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Foreword 


We are delighted to present the first handbook addressing how to integrate empirical 
research (facts) and normative inquiry (values) when assessing health technologies 
(HTA). This handbook is one of the products of the “VALues In Doing Assessments 
of healthcare TEchnologies” (VALIDATE) project. The VALIDATE project was initiated 
to complement how HTA is currently conducted. In current approaches to HTA, 
evidence is often considered value free (neutral), independent from the perspective 
of stakeholders. With the VALIDATE approach we show that facts and values are 


intertwined. This means that HTA should be considered as a type of policy analysis, 
wherein the assessment of safety, clinical and cost-effectiveness of health technolo- 


gies, as well as their wider ethical, legal, and social implications is conducted from 
the view that these aspects are closely interrelated, and wherein stakeholders are 
involved in a more productive way throughout the process of HTA (i.e., an integrative 
HTA approach). To reach this objective, an e-learning course has been developed, 
which will help students to deepen their understanding of the policy and societal 
context of HTA, the role of stakeholders in HTA, and the interplay between facts 
and values in identifying the relevant questions and evidence to be addressed in 
an assessment of health technologies (https://validatehta.eu/e-learning-course/). 
This handbook is a supporting tool for the e-course since it reflects its theoretical 
basis as well as presents pragmatic applications through case-studies. Neverthe- 
less, this handbook provides the necessary information and knowledge to guide 
those who want to embark on an integrative HTA approach, therefore it can be also 
used as Standalone tool. 


This handbook is directed to anyone who wants to understand, initiate, or broaden 
and deepen their knowledge and skills of HTA. Therefore, the targeted audi- 


ence is broad, including Master/PhD students in HTA, health policy and manage- 
ment, health sciences and biomedical sciences, national/regional/academic HTA 
scientists, public health professionals, industry, health services researchers and 
policy makers. 


11 


We very much hope that this handbook will serve as a ready-reference among 


TA doers and users, contributing to improved evidence-informed deliberative 
decision-making. Engaging stakeholders actively in deliberations throughout the 


TA process enables exchange of views on argumentation and evidence. This will 
create an opportunity for more cohesive and inclusive societies. 


The VALIDATE Consortium 


May 2022 
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CHAPTER 1 


First things first 


Abstract 


This first chapter of the handbook introduces you to the background and aims of 
the VALIDATE project and its approach to Health Technology Assessment (HTA). 
After discussing the general challenges related to the implementation and use of 
health technology, and the role of HTA in addressing these challenges, the ideas 
that guide the VALIDATE approach to HTA are introduced. An overview of the hand- 
book is provided, in which these ideas are further developed. This chapter ends with 
introducing the central case study (Non-Invasive Prenatal Testing) that is revisited 
in every chapter to help the reader better comprehend and relate on how the VALI- 


DATE approach is implemented in practice. 


ey messages of this chapter: the VALIDATE approach states that values and norma- 
tive analysis are an integral part of HTA and offers an approach for taking this into 


account. 


After reading this chapter, you will be able to understand the central elements of 
the VALIDATE approach and know in which chapters of this handbook you can find 
elaborations of these elements. 


EU Erasmus+ project VALIDATE 


In the EU Erasmus+ strategic partnerships project “VALues In Doing Assessments 
of healthcare TEchnologies” (VALIDATE), Health Technology Assessment (HTA) 
agencies and academic institutes are collaborating to introduce and train the next 
generation of HTA experts to a novel, integrative approach to HTA. The objective of 


1 For more information about EU Erasmus+ projects: https://ec.europa.eu/programmes/erasmus-plus/ 
opportunities/strategicpartnerships-field-education-training-and-youth_en; more information on 
the VALIDATE project can be found on: https://www.validatehta.eu. 
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the VALIDATE project is to develop an approach to HTA that allows for the integra- 
tion of empirical analysis and normative inquiry, associated teaching materials and 
opportunities for internships for HTA students. To achieve this goal, a consortium of 
seven academic and HTA organizations (Radboudumc, Universita Cattolica del Sacro 
Cuore, Linkóping Universitet, Departamento de Salud, University of Amsterdam, 
Norwegian University of Science and Technology, Hospital Clínic de Barcelona) has 
developed a consensus statement on what knowledge and skills are required to 


conduct this novel type of integrative HTA. Based on the consensus statement, an 


e-learning course and this handbook were developed to educate the next genera- 
tion of HTA experts in conducting integrative HTA. The handbook can be seen both 
as an alternative and a supplement to the e-learning course. 


This handbook will introduce you to the HTA approach developed in the VALIDATE 
project; we explain, and discuss all the elements of the approach, and provide 
examples of how the approach could be used. This first chapter will provide an over- 
view of the VALIDATE approach, the contents of the handbook, and the rationale 
behind VALIDATE. A more in-depth discussion of the philosophy of VALIDATE, that is 
inspired by pragmatism, can be found in Chapter 7 of this handbook. 


Scope of this handbook 


Because the VALIDATE approach is primarily aimed at providing a methodology for 
identifying and addressing normative decisions and issues related to the practice 
of HTA, with a focus on moral and societal values, the discussion of legal aspects 
and frameworks is beyond the scope of this handbook. Although legal frameworks 
set boundaries for the practice of HTA, and it is highly relevant to be aware of them, 
an HTA practitioner still needs skills to address normative questions (because they 
are not always answered or are even raised by legal frameworks) and to identify 
different interpretations of the values that may have been the original impetus for 
a legal framework. 


A task for HTA? 


We are living at a time in which our lives are increasingly technological lives. What 
we do, how we communicate, the way in which we experience and interact with 
the world, is more and more determined by technology. Also, in the domain of 
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health, technology is shaping our way of doing things.? We make use of, among 
others, drugs, devices, vaccines, different ways of organizing healthcare systems, to 
prevent, diagnose or treat medical conditions and promote health. Although these 
technologies enable us to improve our health, they also give rise to complex ques- 
tions concerning the nature of human life, the sustainability and equity of health- 
care systems, and their potential impact on our society. 


Given these extensive uses of health technology, healthcare systems continuously 
need to adapt to the challenges that are associated with it. This implies many deci- 
sions that need to be made, and to introduce more rationality into decision- and 
policy-making the area of HTA emerged as an important scientific response to such 
policy-related endeavours. HTA is a “multidisciplinary process that uses explicit 
methods to determine the value of a health technology at different points in its life- 
cycle. The purpose is to inform decision-making in order to promote an equitable, 
efficient, and high-quality health system” (O'Rourke et al, 2020). 


From its very inception, practitioners of HTA? have been committed to the compre- 
hensive study of the consequences associated with the use of health technolo- 
gies (i.e., the study of a broad spectrum of consequences, ranging from clinical to 
economic and social aspects). This grew out ofan awareness that health technology 
can have profound, unintended, and unforeseen consequences, that need to be 
taken into account when making decisions about the use of such technologies. 
t was recognized that there are different types of impact that need to be addressed 
in HTA: safety, clinical effectiveness, cost-effectiveness, and the ethical, legal, and 


social implications (also known as ELSI) of health technology. Despite this original 
impetus for HTA to perform this comprehensive study of implications of health 


technology, its focus has been often narrowed to issues of affordability (Daniels 
et al, 2016; Lehoux, 2006; ten Have, 2004). Assuming that the budget for health- 
care is limited, and that technology is the main cost-driver in healthcare, HTA is 


N 


Although a general definition of ‘technology’ is hard to give, and there are many discussions on this 
concept and its nature, we will use the term to refer to “The application of scientific or other organized 
knowledge - relating to tools, techniques, products, processes, methods, modes of organisation, or 
systems - to practical tasks”. And a health technology is: “Any intervention developed to prevent, 
diagnose or treat medical conditions; promote health; provide rehabilitation; or organize healthcare 
delivery.” In healthcare, technology encompasses: drugs, diagnostic tests, devices, equipment and 
supplies, medical and surgical procedures, support systems, and organizational and managerial 
systems (source: www.htaglossary.net). 
With ‘HTA practitioner’ we refer to anyone who may be involved in conducting assessments of health 
technology. Given the multidisciplinary nature of this exercise, this may involve people from diverse 
disciplines, ranging from ethics to social sciences to biomedical sciences. 


W 
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often reduced to answering questions like /s this technology effective? Is it safe? Does 
it provide ‘value for money’? Although answering these questions may be necessary, 
it may not cover the broad range of issues and concerns that are raised by health 
technologies and that need to be addressed by decision-makers. What is the value 
of specific innovations? What impact do they have on clinical practice, population health, 
and social development? The importance of these questions is recognized, but in the 
current practice of HTA these questions are often neglected or addressed separately 
as ethical, legal, and social issues (Bellemare et al, 2018; Lehoux et al, 2007). 


Although there can be many practical and methodological reasons for this current 
situation of separate analyses in HTA, one fundamental problem is that the analysis 
of these different types of impact (e.g., safety, clinical effectiveness, cost-effective- 
ness, ELSI) are seen as completely different in their epistemic nature (i.e. ‘episte- 
mology’ is the study of the nature, origin, and scope of knowledge; Refolo, Sacchini, 
2016). This distinction between different types of analysis suggests that there are 


different ways of studying a health technology: 
1. Through empirical research (e.g., studying safety, clinical and cost-effectiveness) 
2. 


Through normative inquiry (e.g., studying ethical, legal, and social implications) 


Consequently, an assessment of aspects such as safety and cost-effectiveness is 
seen as objective and consisting of the collection of empirically testable facts, whereas 
studies of the wider implications (Ethical, legal and social issues) are seen as subjec 
tive and solely involving value judgments (Legault, Suzanne, 2018, Refolo, Sacchini, 
2016). This view makes it very hard to see how HTA could legitimately integrate these 
‘different’ types of inquiry, especially when value judgments may be seen as a threat 


to the legitimacy of HTA itself (Ducey et al, 2017). This also leaves decision-makers 


uncertain as to how, if at all, to take into account ethical issues. 


As a result, health technologies and the practices that are created by them seem to 
be completely distinct from moral values. It is as if they occur in completely separate, 
mutually inaccessible, domains, being divided between a ‘neutral’ domain (e.g., facts 
about consequences of health technology) and a ‘value-laden’ domain (e.g., ideas 
about the (un)desirability of health technology). Consequently, it becomes very diffi- 
cult to answer questions on the desirability of (a particular) using health technology 


that move beyond presuming that this is captured by assessing their effective- 
ness and cost-effectiveness. To be able to answer these broader questions on the 
value of health technology an integrative approach is needed wherein the study 
of safety, clinical and cost-effectiveness of new healthcare technologies and their 
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wider ethical, legal and social implications are closely integrated. Therefore, a critical 
analysis of a strict fact/value distinction in HTA is needed to foster an integrative 
approach to HTA. 


Going back to basics 

Instead of assuming a strict dichotomy between empirical and normative aspects of 
health technology, we propose an approach towards HTA that aims to complement 
the current practice of HTA and allows it to assess health technologies in a more 
inclusive fashion (including different consequences and properties of health tech- 
nology, as well as different stakeholder perspectives). Our goal isto present a frame- 
work that supports a more integrative approach to HTA, wherein the study of safety, 


clinical and cost-effectiveness of new health technologies and their wider ethical, 
legal, and social implications are closely integrated, and stakeholders are involved 
in a more productive way throughout the process of HTA. This could help HTA in 
addressing the type of policy questions that were actually the original impetus for 
developing (health) technology assessment (Lehoux et al, 2007; Lucivero, 2016; 
Torgersen, 2019). This means going back to the roots of HTA, defined as a form of 
policy research (Banta et al, 1993). It should also help HTA to address the needs of 


a population and developments in the organization of healthcare systems related 
to including the patients and citizens perspective in decision-making (Abrishami 
et al, 2017). 


Elements of the VALIDATE approach and preview of this 
handbook 


Overview of this handbook 
The goal of this handbook is to deepen your understanding of the different elements 
of the VALIDATE approach. The structure of this handbook is as follows: 


Part 1: Introduction 
= Chapter 1: First things first (Bart Bloemen & Gert Jan van der Wilt) 
= Chapter 2: The fact/value dichotomy (Pietro Refolo & Dario Sacchini) 


Part 2: Making it work 


= Chapter 3: The method of reconstructing interpretive frames (John Grin) 
= Chapter 4: Scoping (Wija Oortwijn) 


Chapter 1 | First things first 19 


= Chapter 5: Approaches in ethics with a touch of meta ethics (Bjørn Hofmann & 
Lars Sandman) 
= Chapter 6: Context matters (Iñaki Gutiérrez-lbarluzea & Laura Sampietro-Colom) 


Part 3: The philosophy of VALIDATE 
= Chapter 7: A philosophical summary of the VALIDATE approach (Bart Bloemen & 
Gert Jan van der Wilt) 


Part 4: Conclusion 
= Chapter 8: Summing up VALIDATE (Wija Oortwijn & Gert Jan van der Wilt) 


In the following, an overview of the elements of the VALIDATE approach is provided 
together with a preview of the different chapters in this handbook. This will help you 
with getting an idea about what you will learn by reading these chapters. 


The VALIDATE approach 

The VALIDATE approach starts with revisiting the fact/value dichotomy in HTA. 

In contrast to the dominant interpretation described above, our starting point is 

that each type of impact of a health technology have both an empirical and a norma- 

tive (or value) dimension: 

=" Safety: refers to the sort of outcomes that we wish to avoid because of our 
commitment to avoiding harm (the principle of ‘non-maleficence’). 

= Clinical effectiveness: refers to benefits obtained in clinical practice and are 
the sort of outcomes that we wish to achieve because of our commitment to doing 
good (the principle of 'beneficence”), e.g., prolong life, alleviate suffering, restore 
functioning, etc. 

= Cost-effectiveness: refers to the sort of outcomes that we wish to achieve 
because of our commitment to distributive justice/fairness (e.g., proportionality 
between resource commitment and reduction in burden of illness; to ensure 
equitable access to healthcare). 


So, ethical questions concerning the desirability of a health technology are guiding 
the assessment of the impact of a health technology as depicted in Figure 1. 
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SAFETY 


CLINICAL EFFECTIVENESS 


COST-EFFECTIVENESS 


ETHICS 


Figure 1 Normativity of assess 


SAFETY 


> CLINICAL EFFECTIVENESS 


COST-EFFECTIVENESS 


ment aspects 


Therefore, ELSI questions concerning health technology are not in a separate cate- 


gory with respect to the nature of their analysis, and the role of value judgments. And 


in every empirical inquiry, the following questions need to be addressed: (i) What 


sort of things matter to us, and for what reason? (ii) What sort of things may happen 


when we would start using this 
we obtain reliable knowledge? In 


technology? (iii) And, about what sort of things can 
other words, HTA is not a matter of collecting THE 


facts, but a matter of collecting facts that are considered to be: 


= Relevant: which consequences are desirable or to be avoided in view of values 


to which we are committed; 


background theory and/or evi 
specific context? 


Amenable to scientific inqui 


Plausible: which consequences can be plausibly associated (on the basis of 


dence) with the use of a health technology in a 


ry: which consequences can be studied by 


methods of inquiry that are held to produce reliable knowledge and under- 


standing? 


Chapter 2 will provide more 


information about the history of the fact/value 


dichotomy, the criticism it received, its role in HTA, and our reconceptualization 


of the relation between facts and values in HTA. 


HTA as a form of policy 


research 


Another important element of the VALIDATE approach is that we view HTA as a form 


of policy research. Given that HTA aims to provide decision-makers with informa- 


tion on policy alternatives, it should actually be able to address policy-related ques- 


tions that are raised by health 


technology. As already discussed, these questions 


move beyond the affordability of health technologies and demands an integration 


of empirical and normative ana 
needs of a population and decisi 
efficient, and high-quality health 


ysis. This requires that HTA takes into account the 
on-makers, to be able to contribute to an equitable, 


care system (Abrishami et al, 2017). 
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To take this policy context into account, HTA should make use of insights from 
policy sciences. Starting with the realization that there may be different views in 


society and cultures wh 


en it comes to what is regarded as relevant and plausible 


policy options with respect to the use of health technology, we should learn how to 


explore and address thi 


s pluralism. Understanding these different views is essen- 


tial when determining which types of information should be produced in an HTA. 


To explore and understand these views, we propose to make use of the method of 


reconstructing interpretive frames. This method aims to, by involving and interviewing 


stakeholders, reconstruct the conceptual schemes that stakeholders use (often 


implicitly) to make sense of concrete situations and guide their actions. These inter- 


pretive frames consist of t 


he following elements: 


= Judgment of solutions: What is likely to work? Which measures could be helpful 


in addressing this hea 
helpful, and why not? 


feasible in this context 


= Problem definitions: 


th problem? Which measures would probably not be 
s this health technology likely to be effective, appropriate, 


“Y 


What are the key problems of the current situation? Which 


problems are encountered by patients, their families, healthcare providers or 


healthcare managers 
= Background theory: 


in the care of...? 
What are the main causes and mechanisms responsible 


for the current situation? 
= Normative preferences: What is desirable? What is it that we would like to 


achieve and what is it 


that we would like to avoid? 


These interpretive frames can be considered a type of tacit knowledge: they remain 


mostly implicit, but can be made explicit. The method for this is called reconstructing 


interpretive frames. Th 
underlying judgments o 
a critical, constructive s 


e purpose of making factual and normative assumptions 
n health interventions and technologies explicit is to enable 
crutiny of those assumptions and to foster learning among 


stakeholders. Moreover, the output of this reconstruction can be used to under- 


stand the type of policy problem you are dealing with and to make decisions on 


which type of information should be produced or collected in an assessment. 


Chapter 3 will provide more information about the method of reconstructing 


interpretive frames and insights from policy sciences that are relevant to the 


practice of HTA. 
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HTA and the importance of scoping 

When the type of policy problem you are dealing with is clear, you have to trans- 
late this into research questions. This process of translating policy problems into 
research questions is called scoping. To avoid answering the wrong questions, 
neglecting the concerns and values of decision-makers and stakeholders, it is 
important that the HTA community addresses more clearly what matters to the rele- 
vant healthcare system, as well as the relevant stakeholders, and the reasons why. 


This should be done before the start of an actual assessment, and based on this 
the relevant research questions to be addressed should be identified. This could 
ensure the relevance of the information collected during an assessment, and help 
policy-makers in explaining their decisions to the populations they serve. 


Chapter 4 will provide more information about how to define the objectives 
and research questions of an HTA by a systematic exploration of relevant 
aspects of a policy problem and the involvement of stakeholders. It integrates 
the lessons of Chapters 2 and 3: because the relevance of facts depends on 
the adopted perspective, and there are different perspectives in society with 
respect to the desirability of health technology, the objectives of an HTA should 
be defined by taking into account these perspectives. 


HTA and the importance of normative analysis 

An important element in defining the objectives and research questions of an 
HTA is the explication and clarification of what makes a health technology desir- 
able. This helps in identifying relevant outcome measures. For example, the use of 
cost-effectiveness analysis should be supported by ethical arguments that explain 
why maximizing efficiency is a relevant goal to be realized by the use of a health 
technology. The relation between maximizing efficiency and realizing an equitable, 
efficient, and high-quality healthcare system should be made clear. Such a norma- 
tive analysis not only helps in improving the consistency between decisions that 
are informed by HTA, it also prevents that HTA focuses upon aspects that are not 
considered relevant by decision-makers and the population. 


Acknowledging a central role for normative analysis in HTA evokes several ques- 


tions: How can ethical questions be addressed in HTA? Is there more to ethics than just 
opinions? Which methods can be used? 
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Chapter 5 will discuss the possibility of moral knowledge, different approaches 
to normative analysis, and introduces an overview of methods for conducting 
normative analysis in the context of HTA. 


HTA and the importance of context 

When conducting an HTA, you should also be aware that HTA is context-dependent. 
Whether the outcomes of an assessment should inform decisions at the national 
level, regional level, or at the level of a hospital, makes a difference when it comes to 
its objectives and research questions. In addition, the consequences of health tech- 
nology are context-sensitive. The effects of health technology are influenced by the 


needs and demographics of a population, characteristics of a healthcare system, 


characteristics of the technology itself, the legal framework that guides its imple- 


mentation, and cultural and social features of the situation in which it is used. These 


contextual factors all influence the conditions of use of health technology, and 


should be taken into account to increase validity ofthe outcomes of an assessment. 


Chapter 6 introduces you to several aspects of the context of HTA and shows 
the importance of context during the assessment process, and discusses the 
particular perspective of hospital-based HTA. 


The philosophy of VALIDATE 

To conclude, a central tenet of the VALIDATE approach is that HTA is a practice in which 
facts and values meet. To be able to address policy-related questions, HTA should not 
only provide information on the plausibility of claims on potential consequences of 
health technology, it should also assess the desirability of these consequences. This 
requires both an understanding of the technology and its (potential) impact, as well 
as being able to think about this impact in light of a set of norms and values. As stated 
before, this means that normative analysis is an intrinsic part of HTA. 


To propose such a central role for normative analysis in HTA may evoke a critical 
response. A widely held view - also within the context of HTA - is that there are 


no ways of rationally resolving disagreements on normative issues. This position 
is called moral scepticism, which refers to a diverse collection of views that deny or 
raise doubts about various roles of reason in morality. If we are not able to offer 
a response to this position, the consideration of ethical issues in HTA is seriously 


hampered. Therefore, we need to rethink our position towards the nature of ethical 
issues in HTA. 
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Chapter 7 will elaborate our view on the nature of ethical issues in HTA, 
presenting a response to moral scepticism. This response is based on the philo- 
sophical tradition of pragmatism. After reading Chapter 7, you will understand 
the relation between pragmatism and the VALIDATE approach. 


The handbook will end with a summarizing chapter, and a glossary of specific terms 
used. For those of you not very familiar with the topics discussed in this handbook 
itis highly recommended to have a look at the glossary. 


Case study: Non-Invasive Prenatal Testing 


To help you understand the VALIDATE approach, and relate the different elements 
to a real-world example, every chapter will return to the case study of Non-Invasive 
Prenatal Testing (NIPT). Itis an example of a health technology that may be used in 
many different ways and, therefore, raises different ideas about its desirable use. In 
the following sections, we will shortly introduce this case study by providing back- 
ground information and highlighting important issues related to assessing NIPT. 


Prenatal screening 

In many countries, pregnant women are offered different screening tests during 
pregnancy to identify any complications, diseases, or disorders at an early stage. 
The primary purpose of screening is to detect early disease or risk factors for 
disease in large numbers of apparently healthy individuals. Therefore, a major 
difference with diagnostic tests is that screening tests can be offered to individual 
who have no symptoms or any or any prior reasons for suspecting a higher 


chance at disease. In the case of prenatal screening tests, this means that they 
can be offered to any pregnant women irrespective of any prior knowledge on an 
increased risk of their unborn child having any medical conditions (e.g., based on 
age or genetic conditions in the family). This differs with available diagnostic tests 


that only offered based on a certain form of suspicion that the unborn child will 
have a specific medical condition. 


Besides blood tests that screen for infectious diseases and provide information on 
blood group, and a structural ultrasound scan that tests for anatomical anomalies, 
an important category of prenatal screening tests is those that aim to detect foetal 
chromosomal abnormalities. These tests identify genetic conditions of the unborn 
child, often focusing upon trisomy 13 (Patau syndrome), 18 (Edwards syndrome), 
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and 21 (Down syndrome). Until recently, the screening strategy for these conditions 
consisted of offering combined first-trimester screening (CFTS) which entails an ultra- 
sound examination and testing blood serum markers that collectively estimates the 
risk of a trisomy. After a positive test result (increased risk of trisomy), prospective 
parents where offered the choice to take an invasive prenatal test (e.g., amniocen- 
tesis, chorionic villus sampling) to confirm diagnosis or refrain from further testing 
(Hui et al, 2017). 


Non-Invasive Prenatal Testing (NIPT) 

In recent years, Non-Invasive Prenatal Testing (NIPT) has rapidly transformed the 
prenatal screening landscape. Following the discovery of cell-free foetal DNA, NIPT 
was introduced in clinical practice in 2011 and became increasingly available to preg- 
nant women worldwide (Gadsboll et al, 2020, Hui et al, 2017, Van der Meij et al, 2019). 
NIPT is a prenatal screening procedure that analyses cell-free foetal DNA, which 
circulates in the mother's blood during pregnancy, in order to obtain information 
about the foetal genotype. Because only a blood sample of the mother is needed, 
the procedure does not pose any additional risks of a miscarriage that are asso- 


ciated with the available invasive tests (amniocentesis, chorionic villus sampling). 
Besides this reduction in risk of procedure-related miscarriage it has other potential 
advantages. It can be performed relatively early during pregnancy (around the 10" 
week of pregnancy), it could have a higher reliability than existing tests, and could 
potentially be used to analyse the entire foetal genome which provides the option of 
detecting conditions for which no other screening protocol exists. 


The integration of NIPT in clinical practice is often being performed in a stepwise 
manner. For example, in the Netherlands the Dutch NIPT consortium (a multidisci- 
plinary collaborative partnership among different stakeholders involved in public 
prenatal care) was granted a governmental license to introduce NIPT in the context 
of an implementation study trial (Trial by Dutch Laboratories for Evaluation of Non- 


Invasive Prenatal Testing, TRIDENT). In TRIDENT-1, starting in 2014, women with a 


positive combined test result could choose between NIPT or invasive diagnostic 
testing. In April 2017, a follow-up study started (TRIDENT-2), in which al/ pregnant 


women are offered the choice between a combined test or NIPT (Van der Mejj et al, 
2019), see also Figure 2. A unique feature of TRIDENT-2 is that women who elect NIPT 


can choose a report on chromosomes 21, 18 and 13 either with or without infor- 
mation on the other autosomal chromosomes (i.e., excluding sex chromosomes). 
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Figure 2 Changing prenatal screening practice in The Netherlands 


Potential uses of NIPT 

In implementing NIPT, countries make many different decisions with respect to 
whether it is offered commercially or as part of a national screening program, 
whether it is offered to all pregnant women or only for women with a particular 
risk, different ways of reimbursement, and which genetic conditions to screen for 
(Gadsboll et al, 2020). Especially the aspect of the scope of NIPT (i.e., which genetic 
conditions are screened) leads to many decisions and debate. Because NIPT could 
potentially offer genome-wide information, there are many potential models of 
offering NIPT ranging from screening on specific genetic conditions to proving 
whole-genome coverage. This stirs debate on whether information on risk factors 
for late-onset disease, abnormalities related to a mild phenotype, gender, or genetic 
information on non-clinical conditions should be provided. 
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Not only the desirable scope of NIPT raises many policy questions, but also its 
potential use in combination with other health technologies. Because NIPT could 
potentially offer information on the risk of complications during pregnancy, or 
conditions of the unborn child that need treatment, it could be used to decide 
upon, and guide, prenatal treatment (e.g., drugs, surgical procedures, gene therapy). 


Therefore, although NIPT is often classified as a screening test, it could also be seen 
as a treatment strategy. This classification of NIPT may influence the comparators 
(i.e., Comparing it to other prenatal tests or to other ways of treatment) and criteria 


(i.e. what are the potential benefits?) that are being used to assess its potential 
value. The relevance of this issue can be seen in the case of the introduction of 
NIPT in Germany, where the decision of the Federal Joint Committee (Gemeinsamer 
Bundesausschuss), the supreme decision-making body on determining therapeutic 
usefulness, cost-effectiveness and medical necessity of a health technology) to 


define NIPT as a medical procedure was heavily contested (Braun et al, 2018). 


The need for an integrative assessment 

IPT is an example of a novel and morally challenging technology that raises ethical 
questions related to its desirable use and evokes issues relevant to people beyond 
those who interact with the technology directly. As already mentioned, one impor- 
tant challenge with respect to NIPT is that its desirable scope and purpose can be 
conceptualized in many, and sometimes contradictory, ways. It can be seen as a 


technology that only provides information on genetic abnormalities, a technology 


that helps preventing the birth of children with genetic abnormalities, or a technology 
that is able to enhance reproductive autonomy. These different views do not only lead 


to conflicts between stakeholders, but also to contradictory statements/conclu- 
sions between different components of an assessment. Economic analyses of NIPT 
make use of outcome measures such as cost per additional abnormality detected, 
or cost savings per disabled child not born, that frame NIPT as a technology that 
becomes more cost-effective when it prevents a sufficient number of births affected 
by genetic disability. This may suggest a purpose of NIPT that conflicts with the 
desirable purpose which is suggested by social and ethical analyses (Kibel et al, 
2017). This fragmented, and potentially conflicting, assessment of different types of 
impact of NIPT puts decision-makers in a challenging situation. Therefore, it would 
be highly relevant to be able to conduct an integrative assessment in which the 
clinical, economic, social, and ethical aspects of NIPT are comprehensively assessed. 
In other words, an assessment of NIPT highlights the need for the VALIDATE move in 
which normative and empirical analysis are integrated. 
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Value judgments in assessments of NIPT 

When conducting an assessment of NIPT, value judgments are inevitable. For 
example, when assessing the effectiveness of NIPT, questions are raised with 
respect to which outcomes can be regarded as desirable. This may not be limited 
to health-related outcomes, and may need the use of normative concepts such as 
reproductive autonomy (Kessels et al, 2019). And even when standard health-related 
outcome measures are used, like the Quality-Adjusted Life Years (QALY), norma- 
tive questions need to be addressed: “Whose health benefits should be counted: 


those of the unborn child of those of the prospective parents? Are QALYs lost when 
a woman decides, after a positive NIPT test result, to abort a foetus with Down 
syndrome?” (Goldhaber-Fiebert et al, 2015). 


The VALIDATE approach 

The above discussion on NIPT illustrates the central role of normative analysis in 
HTA. To be able to offer information on potential consequences of NIPT and combine 
information on different types of consequences in a comprehensive assessment, 
normative and empirical analysis should be integrated. The VALIDATE approach 
argues that this integration is best achieved by working out the desirable outcomes 
that are sought by the use of NIPT. In that way, we can understand and discuss how 
certain facts can be collected and be relevant in an assessment of NIPT. By making 


explicit the relation between choices that are made in conducting the assessment 
and the, collectively determined, desirable ends of NIPT, the assessment becomes 


more transparent and meaningful (Van der Wilt, 2017). 


Working out the desirable ends of NIPT, in collaboration with stakeholders, helps in 
identifying the different views in society (interpretive frames, Chapter 3), defining the 
objectives and research questions of an assessment (Scoping, Chapter 4), addressing 
normative questions (Chapter 5), while highlighting and taking into account relevant 
contextual factors (Chapter 6). These chapters will return to the case of NIPT to 
further illustrate these elements of the VALIDATE approach. After discussing the 
philosophy of VALIDATE (Chapter 7), the final chapter (Chapter 8) will summarize the 
VALIDATE approach and return to NIPT. 
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CHAPTER 2 


The fact/value 
dichotomy 


Abstract 


This chapter is focused on the so-called “fact/value dichotomy”. In the current prac 
tice of Health Technology Assessment (HTA), value judgments are often considered 
external to HTA or separate from it and are addressed by experts (in ethics) and 
decision-makers after the “real” HTA is finished. In other words, facts and values 
seem to be two completely different “substances” that come from two completely 
different sources. Is this dichotomy attainable? Is it desirable? This chapter will argue 
that a strict fact-value distinction is not so easy to maintain, based on insights from 
Philosophy and HTA practice. 


Key messages of this chapter: HTA cannot be considered a matter of collecting the 
facts about a certain health technology, but rather “a matter of collecting facts that 
are considered plausibly associated with the use of the technology, relevant, and 
amenable to accepted methods of scientific inquiry”. 


Having read this chapter you should be able to understand that facts and values are 
intertwined in HTA, even though the latter may remain implicit or tacit. 


Introduction 


In Chapter 1, it has been noted that health technologies and the practices that 
are created by them seem to be completely insulated from moral values. In addi- 
tion, an assessment of aspects such as Safety and cost-effectiveness is often seen 
as objective and consisting of the collection of empirically testable facts, whereas 
studies of the wider implications (ethical, legal and social issues) are seen as subjec 
tive and solely involving value judgments. This situation is well-described by Gorski 
(2017): “Imagine a large, double-sink such as you might find in the basement of a 
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single-family home somewhere in North America. An impermeable wall separates 
the two sinks. Each sink is serviced by its own spigot. The righthand spigot emits a 
red tinctured liquid called ‘values’. The piping from this spigot seems to go down 
through the floor and into the ground to a deep-water well, labelled ‘emotion’ (or is 
it ‘conviction?’). The left-hand spigot emits a blue tinctured liquid called ‘facts’. The 


plumbing for this spigot seems to go out through the exterior wall to a surface-water 


in 


source called ‘observation’. 


This is more or less how some people interpret the relationship between facts (what 
is) and values (what ought to be). For them, facts and values are two completely 
different “substances” that come from two completely different sources. Facts (e.g., 
the earth revolves around the sun) are inarguable truths of the external world, given 
through the observation of nature, and they are objective. They could be ordered 
into theories and explanations using science’s methodological tools (Gorski, 2017). 
On the other hand, values (e.g., to be honest) are personal beliefs that motivate 


people to act one way or another, cannot be derived from empirical observations, 
and they are subjective. Unlike facts, they cannot be proven true or false by any sort 
of scientific method. 


This seems to happen also in the current practice of Health Technology Assessment 
(HTA): facts (e.g., data on safety) are being collected and processed in order to inform 
regarding possible or likely consequences of using a certain health technology. 
When the empirical evidence base seems to be sufficiently robust and comprehen- 
sive, then the issue about its value (ethical analysis) is raised separately: given these 
facts, how is it possible to judge the overall value of this health technology? 


However, this way of proceeding, apparently so obvious and easy, hides some 
snare. Indeed, the relationship between facts and value - better known as “fact/ 
value dichotomy” - is one of the most debated topics in Philosophy. It has been 
addressed in many ways: one debate asks whether it would be possible “to draw” 
values (moral norms) from the knowledge of nature; another discussion focuses 
on studying the relationship between descriptive propositions (e.g., the moon is 
spherical) and normative propositions (e.g., killing is wrong); another one considers 
whether ethics is an authentic form of knowledge. 


In this chapter, two questions will be addressed with respect to this fact/value 


dichotomy: 1) Is it attainable? and 2) is it desirable? The chapter will argue that a strict 
fact-value distinction is not so easy to maintain, based on insights from Philosophy 
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and HTA practice. Based on this discussion, an analysis about the consequences of 
this conclusion for HTA will be performed. The text will be supported by the discus- 
sion of two case studies, 


The value/fact dichotomy 


Origins of the fact/value dichotomy* 

At the centre of much of the debate around the value/fact dichotomy there is a 
famous passage from the Scottish philosopher David Hume (1711-1776): “In every 
system of morality, which | have hitherto met with, | have always remarked, that the 
author proceeds for some time in the ordinary ways of reasoning, and establishes 
the being of a God, or makes observations concerning human affairs; when all of a 
sudden | am surprised to find, that instead of the usual copulations of propositions, 


is, and is not, | meet with no proposition that is not connected with an ought, or an 


ought not. This change is imperceptible; but is however, of the last consequence. For 
as this ought, or ought not, expresses some new relation or affirmation, ‘tis neces- 
sary that it should be observed and explained; and at the same time that a reason 
should be given, for what seems altogether inconceivable, how this new relation can 
be a deduction from others, which are entirely different from it. But as authors do 
not commonly use this precaution, | shall presume to recommend it to the readers; 
and am persuaded, that this small attention would subvert all the vulgar systems of 
morality, and let us see, that the distinction of vice and virtue is not founded merely 
on the relations of objects, nor is perceived by reason” (Hume, 1739). 


The passage is also known as “Hume's law”, “Hume's guillotine” or as “the is-ought-fal- 
lacy”, and has been interpreted in several ways. In short, it states that one cannot 
logically derive an “ought” from an “is”, that is, one cannot derive “norms” from 
“facts” - there is no logical bridge between fact and value. 


A similar argument has been defended by the English philosopher George Edward 
Moore (1873-1958). In his “Principia ethica” (Moore, 1903), he argued against any 
identification of moral properties with natural properties. In particular, he argued 
against what he called the “naturalistic fallacy” in ethics, by which he meant any 


attempt to define the word “good” in terms of natural qualities. 


4 Some parts of this text are taken from Refolo et al, 2016. 
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The German sociologist Max Weber (1864-1920) arrived at similar conclusions too 
(Weber, 1949): he made a strict distinction between “statements of facts” (describing 
reality) and “statements of value” (relatingto an ideal). The former ones were consid- 
ered objective and the latter subjective. Given this, he argued that Science, as the 
realm of facts, must be seen as strictly separated from the realm of values, i.e., 
ethics, aesthetics, and politics. 


Despite some differences in terminology, all the aforementioned authors made a 
clear distinction between facts and values and repudiated any attempt to derive 
moral values from facts. 


The fact/value dichotomy in Philosophy of Science 

A version of the fact/value dichotomy was also defended in Philosophy of Science, 
the academic discipline that deals with questions about what Science is, how it works 
and the methods through which it produces reliable knowledge.? An important 
movement of the early 1900s, Logical Positivism (also known as Logical Empiricism 
or Neo-positivism), stated that Science is a formal activity that creates and accumu- 
lates knowledge by directly observing, and confronting (i.e., by doing experiments), 
the natural world. Science tries to discover how the world really is by observing it 
carefully. Hypotheses and theories are generated and tested based on these obser- 
vations, and only when theories make predictions that can be verified - correspond 
to how the world really is (as known through senses and the use of scientific instru- 
ments) - they can be seen as true. This view on Science is known as Logical Posi- 
tivism because its central claim is that the only source of authentic knowledge is 
sensory experience or observations, and inferences that can be drawn from these 
observations by using logical techniques. The term ‘Positivism’ derives its meaning 
from the French word positivism, which is derived from positif meaning: ‘imposed on 
the mind by experience’. 


The aim of Logical Positivism was to construct scientific laws and theories to describe 
and express relationships between observable phenomena. It can be interpreted as 
a program of radical “re-foundation” of knowledge exclusively based on logics and 
observations, which should have led to the elaboration of a “unified language” for 
Science as a whole. Core notions and assumptions of Logical Positivism were: 


5 For more background information about Philosophy of Science, see UC Museum of Paleontology of 
the University of California at Berkeley, “The philosophy of science”, URL: https://undsci.berkelye.edu/ 
article/philosophy. 
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1. there is a division between the real world, “the given”, and its perceptions and 
expression by humans; 

2. the real world, “the given”, can be grasped by empirical sciences; 

3. the truth, established by empirical sciences, can be expressed in protocol 
sentences (i.e., statements that describe immediate experience or perception) 
and observation reports; 

4. protocol sentences and observation reports can be understood, described and 
conceptualized by logical analysis, thereby bridging the division between the 
world and the words, “the given” and the concept, facts and theory (Burkard, 
2018). 


The assumption on which the whole philosophical conception of Neo-Positivism was 
based is the well-known “theory of verification”, according to which a proposition is 


i 


“cognitively meaningful” only if some finite procedure conclusively determines its 
truth. This signifies that any statement that cannot be verified by an empiricist crite- 
rion is meaningless. Metaphysics, ontology, as well as ethics fail this criterion, so 
they are cognitively meaningless. In this way, Neo-Positivism celebrated a sort of 


“divorce” of Science from Ethics and facts from values. 


Challenging the fact/value dichotomy 
However, from the 60s in the 20" century, the fact/value dichotomy as well as the 
neo-positivistic approach were criticized in many ways and they lost appeal. 


In this context, the “philosophical hermeneutics” initiated by Martin Heidegger 
(1889-1976) and developed by Hans-Georg Gadamer (1900-2002) in his “Truth and 
Method” (1960) came to play a fundamental role. In essence, Gadamer claimed that 


understanding is never fixed and final but always steeped in language, dynamic 
and fluctuating. “Prejudices” are the very source of our knowledge. Hence, under- 
standing is always personal, human, and subjective. This imply that at work there 
are always different perspectives or - as they will be called in Chapter 3 - different 
“interpretative frames”. 


This assumption had a significant impact on the methodological presuppositions 
on which modern Science was built. According to the new viewpoint, there can be 
no universal standpoint from which “objective knowledge” can be achieved, and 
all understanding - including scientific understanding - has to be considered as 
“contextual” and “historical”, 
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Another important role has been played by the German philosopher Karl-Otto Apel 
(1922-2017). He pointed out that moral language analysis always requires a crite- 
rion to distinguish moral language from any other form of language. This helped to 
support the thesis that existence of merely descriptive propositions (as scientific 
propositions) is an illusion, and that itis impossible to separate the normative dimen- 
sion from a descriptive one. Moreover, Apel argued that the fundamental shortcom- 
ings of Positivism spring from a lack of reflection “upon the fact that all cognition of 
objects presupposes understanding as a means of intersubjective communication” 
(Apel, 1972). Science is unintelligible as human activity, if one cannot understand the 
implicit and explicit conventions and notions, or more general, the communication 
community or language game, which it presupposes. Even tacit conventions about 
the use of words, not to mention explicit conventions about definitions, theoretical 
frameworks, or statements of facts in empirical science, imply “an intersubjective 
consensus about situational meanings and aims of practical life” (Apel, 1972). 
t follows that objective Science presupposes hermeneutics (or pre-hermeneutics). 


n this sense, Science is not merely objective, but a “human enterprise”. 


Finally, the American philosopher Hilary Putnam (1926-2016) has recently traced the 
“collapse of the fact/value dichotomy”. According to him, there is a distinction to 


be made, useful in some contexts, between statements of fact and statements of 
value, especially of ethical value. Nevertheless, a strict dichotomy between fact and 
value would be indefensible because on the one side normative (e.g., ethical and 
aesthetic) judgments have always a factual basis, and on the other side scientific 


judgments encompass normative elements. Consequently, Science cannot be 
considered as “value-free”, since “Science itself presupposes values which are in the 
same boat as ethical values with respect to objectivity” (Putnam, 2002). 


These and other reflections have considerably reduced the significance of the fact/ 
value dichotomy. 


Although the clear distinction between facts and values is nowadays difficult 
to defend philosophically, it still appears to be strongly present in the common 
mentality. Only scientific facts are often believed to be worthy of “serious” consid- 
eration, whilst value judgments are frequently considered as subjective, relative, to 
some extent unreliable, and anyway external to scientific speech. 
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The value/fact dichotomy in HTA 


This seems to happen also in the current practice of HTA: value judgments are 
often considered external to HTA or separate from it and are addressed by experts 


\" 


(in ethics) and decision-makers after the “real” HTA is finished. This conception of 


value judgments in HTA is illustrated in Figure 1. 
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Figure1 The conventional concept of HTA — Conventional HTA mainly focuses 
on the collection of quantifiable data, whereas ethical, social, cultural, and legal 
issues, including values related to the use of health technologies, are addressed 
independently and at a late stage in the HTA process. In this view, the assessment 
stage involves the collection of factual information, whereas values only enter the 
process when interpreting this information and taking into account wider aspects to 


make a recommendation (the appraisal stage). 


An HTA usually starts by presenting a more or less comprehensive description of 
the sort of consequences that may be expected from the wider adoption of a health 
technology (Van der Wilt, 2016). Under this view, the first step of an HTA process 
consists in conducting an empirical inquiry of a health technology's safety, clinical 
effectiveness, and cost-effectiveness. This phase involves the analysis of evidence, 
which explains the strict correlation between HTA and Evidence-Based Medicine 
(EBM) (Luce et al, 2010). Subsequently, it is asked whether any ethical issues may 
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be anticipated, given some set of ethical principles, such as beneficence, non- 
maleficence, autonomy, and justice. 


What it has been described here also reflects well the current distinction between 
assessment and appraisal in the context of HTA. HTA agencies typically separate the 
assessment from the appraisal phase in the HTA process. “Assessment” refers to data 
collection that would be useful to form a basis for decision-making. “Assessment” 
differs from “appraisal”, which generally implies some form of recommendation 
(normative level) about the implementation/non implementation/disinvestment of 
a health technology based on a previous assessment. Such a recommendation can 
led to several concrete actions: encouraging, discouraging or even prohibiting imple- 
mentation, reimbursing, funding, disinvesting, etc. Some HTA agencies are restricted 
to assessments only and do not make recommendations about implementation/not 
implementation/disinvestment in the healthcare system, while others perform both 
assessment and appraisal (Sandman et al, 2014). 


A challenge with this approach assumes that it is possible to separate facts (empirical 


inquiry of a technology’s safety, clinical effectiveness, and cost-effectiveness) from 
values (normative dimension). As such, the former are considered objective truth 
claims (they would tell something about some part of reality), whilst the latter would 
be subjective, and would come into play only after the facts have been collected. 


As outlined above, a clear distinction between facts and values is difficult to maintain 
in the current panorama of Philosophy. It is, in fact, recognized that value frameworks 
are already operative at the stage of facts collection. The fact is that many of them 
remain implicit or tacit may reinforce the illusion of a certain scientific objectivity. 
Actually, empirical inquiries of a health technology's safety, clinical effectiveness and 
cost-effectiveness, which are often considered objective, always presuppose norma- 
tive assumptions, and are value laden. They presuppose a valid hermeneutical prism, 
i.e., value judgments about which facts have to be considered relevant and which facts 
do not have to be considered relevant. Without a previous (usually implicit) “inter- 


subjective consensus” (Apel, 1972) about them, no assessment would be possible. In 
this sense, “effectiveness, safety, cost-effectiveness which many consider assertions 


are (...) relative evaluations. In each case, the result of the analysis attributes a quality 
to a health technology using criteria. For each quality attribution there is a choice of 


criteria, a priority setting and a mode of gathering information to apply the criteria to 
the technology at stake” (Legault et al, 2018). 
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An example could help to clarify this point. A very common practice in current HTA is 
the calculation of the incremental costs that are incurred by adopting a novel health 
technology and the additional quality-adjusted life years (QALYs) that may be gained, 
as compared to usual care. These facts derive their relevance from utilitarian theory, 
stipulating that justice requires that for every gain in QALY a proportional amount 
of resources should be spent. However, if the consequences of adopting a health 
technology were assessed from the perspective of a different concept of justice 


(value judgment), such as egalitarianism, those calculations would be irrelevant, and 


completely different information would be necessary (Van der Wilt, 2016). 


Therefore, empirical analysis is always guided by normative assumptions. To be more 
precise, the relationship between these two dimensions (empirical and normative) is 
a relation of plausibility, relevance, and amenability. 


Plausibility refers to the fact that the only consequences investigated are those of 
adopting a health technology that are believed to be plausible. For instance, when 
someone is faced with the task of assessing a drug, there are some things that 
strike as potentially plausible (e.g., the patient will take the medication) while others 
as potentially implausible (e.g., the patient will always throw the medication in the 
garbage). In this sense, plausibility is a function of knowledge and understanding 
of the technology and the sort of impact it may have. In short, plausibility refers 


to the question: what sort of effects are likely to occur, in view of knowledge and 


understanding of the technology, the process in which it intervenes, and its mode 


of action? 


As outlined above, relevance is a function of norms and values. The commitment 
to norms such as the promotion of a certain conception of justice, or the relief of 
suffering, the avoidance of harm, or respecting autonomy determine the sort of 
data (outcomes) that become of interest. In short, relevance refers to the question: 
what sort of effects are anticipated that are not only likely to occur, but that will also affect 
our judgment of the technology under scrutiny? 


The two dimensions - what sort of outcomes are plausible given the current knowl- 
edge of mechanisms involved, and what sort of outcomes are relevant given the 
values that are considered as important - are always operative in HTA. Moreover, 
they work in a concerted way: generally, features that are considered relevant but 
unlikely to materialize are not pursued. Likewise, features that are considered plau- 
sible to occur but of little relevance are not pursued. 
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Finally, amenability refers to the fact that it is possible to investigate those conse- 
quences of adopting a health technology that are amenable to some mode of empir- 
ical inquiry. For instance, if there is no chance for determining how many patients 
will not take their medication, an assessment of this aspect will not be possible. 
In this sense, amenability to scientific inquiry is a function of our methodological 
and epistemological standards. In short, amenability refers to the question: ¡s this 
research feasible? 


Value judgments permeate all levels of HTA. It is beyond the scope of this handbook 
to show all these interlacements, which are better addressed by specific contribu- 
tions (Hofmann et al, 2014; 2018). To give some examples, it is possible to mention 
the relationship between empirical inquiry and normative dimension, as outlined 
above; or the assessment of ethical, social, legal aspects of a health technology, 
which are obviously value related; or the HTA itself which is clearly evaluative. More- 
over, there are many levels of values at play in the HTA process, for example, in the 
selection of endpoints or in selection/presentation of HTA results. There are stake- 
holder involvement processes with different value perspectives, or other values 
assessment frameworks that have been developed for HTA, such as multi-criteria 


decision analysis (MCDA), and so on. A further issue concerns what values are 
currently taken in account in HTA studies, what is the hierarchy among these values, 


and what it is the relation among these values. 


However, in a more general way, it is possible to say that there are normative aspects 
in and of HTA(Van Oudheusden et al, 2019). 


In conclusion, it is possible to affirm that facts and values are intertwined in HTA, 
even though the latter may remain implicit or tacit. In the light of this perspective, 
HTA cannot be considered a matter of collecting the facts about a certain health 
technology, but rather “a matter of collecting facts that are considered plausibly 
associated with the use of the technology, relevant, and amenable to accepted 
methods of scientific inquiry” (Reuzel et al, 2000). This should not come as a surprise 
as HTA is a type of evaluation (Hofmann et al, 2018). 


In the next paragraph, the case cochlear implants for prelingually deaf children will 


be discussed in order to illustrate the relevance of normative assumptions in HTA 
(Van der Wilt et al, 2000). A similar analysis will be performed on the case study of 
Non-Invasive Prenatal Testing (NIPT) (See Box 1). 
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Case study: Cochlear implants for prelingually deaf 
children 


A cochlear implant is a small, complex electronic device that can help to provide 
a sense of sound to a person who is profoundly deaf or severely hard-of-hearing. 
The implant consists of an external portion that sits behind the ear and a second 
portion that is surgically placed under the skin. An implant does not restore normal 
hearing but can give a deaf person a useful representation of sounds in the environ- 
ment and help him/her understand speech. 


The cochlear implants emerged in the 1970s. Gradually, the group for whom the 
technology was deemed potentially appropriate was expanded from late-deafened 
adults to children, and then to prelingually deafened children, who, by definition, 
have never acquired spoken language in the natural manner. 


A lively debate has developed between proponents of cochlear implants for prelin- 
gually deaf children and some deaf activist leaders. The disagreement has become 
heated with both sides accusing the other of being unethical. 


On the one hand, deaf world representatives define the profoundly deaf as a sepa- 
rate culture (minority culture) from the mainstream hearing society. In particular, 
the term “deaf world” would define “a grouping of deaf persons, usually profound, 


who view themselves as a minority culture with its own customs, values, attitudes, 
knowledge, and language. With the latter characteristics, they claim to have the 
ability to self-construct as a culture” (Gonsoulin, 2001). The common denominator is 
deafness, the sign language is their defining language. 


In turn, a sign language is a visual language based on body gestures instead of sound 
to convey meaning. Sign languages are not pantomime, nor are they a visual rendi- 
tion of the related verbal language: they are full-fledged languages like any other 
human languages. Wherever communities of deaf people exist, sign languages 
develop. 


Consequently, deaf world representatives strongly reject the concept of deafness as 
a disability but claim deafness is a culturally defining characteristic. They challenge 
the performance of cochlear implants in children as a matter of ethical debate. 
The main concern regards the diminution of a minority culture: “Should cochlear 
implants work effectively and be widely used, membership in the Deaf World would 
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decrease significantly over time” (Gonsoulin, 2001). An underlying assumption is 
that preserving minority cultures is a positive value for society as a whole. 


On the other hand, proponents of cochlear implants maintain that deafness is a 
disability, i.e., the failure to achieve an expected level of function. For them, the key 
(and obvious problem) is that deaf children cannot hear and, therefore, they will be 
unable to develop spoken language cognition, emotion, etc. Since cochlear implants, 
when followed by appropriate rehabilitation, restore, albeit imperfectly, hearing to a 
certain extent, they will also support these other developments (Van der Wilt, 2016). 


An underlying notion is that of normal functioning, and the importance of main- 
taining or restoring normal functioning, both for individual and for society at large. 


When facing the task of evaluating the cochlear implants for prelingually deaf chil- 
dren, the current practice of HTA requires that the assessment of the technology 


must be conducted along two lines. On the one hand, empirical evidence is collected 
on the impact of the device on the recipient's hearing capacity, development of 


spoken language, ability to pursue mainstream education, quality of life, etc. On the 
other hand, ethical issues are explored, notably the resistance from deaf organiza- 
tions, claiming that the technology is a threat to deaf culture (Van der Wilt, 2016). 


Following VALIDATE's perspective, HTA is more than a collection of facts; facts 
and values are intertwined in HTA. Therefore, it should not be concluded that 
there are certain facts about the technology on the one hand (e.g., its impact on a 
child's proficiency in spoken language) and some ethical issues on the other hand 
(e.g., the demise of deaf culture). Rather, it is by virtue of normative assumptions 
that the collected facts can be considered relevant to the evaluation. In fact, deaf- 


ness may be framed as a Separate culture, or a disability. In the former case, imple- 


menting cochlear implants is not a solution, and their assessment will have a certain 
relevance; in the latter case, implementing cochlear implants is a solution, and 
their assessment will have another relevance. Under this perspective, any HTA is a 
specific collection of facts, driven by specific normative assumptions, not a collec 
tion of facts that serves as a basis to make up our mind about the comparative value 
of the technology under investigation (Van der Wilt, 2016). 


It is significant that - as reported by Lehoux et al (2000) - in 1997 the Dutch Minister 
of Health decided, very unusually, against the assessment-based advice of the 
Health Insurance Council that would have provided reimbursement of paediatric 
implantation on a normal basis. The Dutch Minister's decision was taken after 


4A The VALIDATE handbook 


consultation with the deaf community and with the organization of parents of deaf 
children. Likewise, in 1990 the U.S. Food and Drug Administration (FDA) approved 


the marketing of the Nucleus 22 channel cochlear prosthesis for surgical implanta- 


tion in children aged two to seventeen years, while in a subsequent position paper, 
the National Association ofthe Deaf (NAD) argued that the FDA had made a number 


of mistakes, concluding with a recommendation that the F 


approval and revise the procedures emp 


The example shows how neglecting normative assumption 


sions of the evaluation are the result o 


in which current practice of HTA proves to be insufficien 


oyed” (NAD, 1991). 


fore, the so-called “scientific facts” cann 


evels of HTA (see Figure 2). 


two different value assumptions. 1 


DA “withdraw marketing 


s can lead to a situation 
t. The different conclu- 


There- 


ot be considered as “objective”, and they 


reflect understanding of the value and meaning of the particular health techn 
n other words, normative assumptions always guide the collection of inforn 


ology. 


nation 


needed to assess whether a health technology is able to realize certain effects. In 
the light of this perspective, Figure 1 appears to be different, values permeate all 


The next questions may be: how to proceed in such case? How to proceed when 


different normative assumptions are at work? One obvious strategy would be to 


involve various stakeholders in the HTA process (see Chapter 4). 
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“FACTS” “VALUES” 


Values 
(e.g., preferences, 
equity, fairness) 


Scientific 
Evaluation 


Conventional HTA decision process 


Data 
(Research) 


Stakeholder involvement 
(e.g., patients, hospitals, payers, clinicians) 


exchange 


Values (e.g., prefer- 
ences, equity, fairness) 


exchange 


Data (Research) 


VALIDATE: integrative HTA process 


Analysis Recommendation 
(Assessment) (Appraisal) 


Figure 2 The conventional HTA process vs. the integrative approach of VALI- 
DATE — VALIDATE proposes an integrative HTA approach, where the safety, clinical 


effectiveness, and cost-effectiveness of health technologies are thoroughly integrated 
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with their wider ethical, legal, cultural, social, environmental, and organisational impli- 
cations and stakeholders are involved in a more meaningful way during the entire HTA 


process. 


Conclusion 


The relationship between facts and value - better known as fact/value dichotomy - 
is one of the most debated topics in modern Philosophy. Even though the clear 
distinction between facts and values is still present in the common mentality, it is 
now philosophically indefensible for several reasons. 


Therefore, the current practice of HTA, for which value judgments are considered 
external to HTA or separate from it, should be overcome. lt ignores the fact that 
normative assumptions are operative at all levels of HTA. 


In the light of this perspective, HTA is not a matter of collecting the facts about a 
certain health technology, but rather “a matter of collecting facts that are consid- 
ered plausibly associated with the use ofthe technology, relevant, and amenable to 
accepted methods of scientific inquiry” (Reuzel et al, 2000). 


Box 1 Case study: Non-Invasive Prenatal Testing 

One way of conducting the evaluation of Non-Invasive Prenatal Testing 
(NIPT) consists in performing an empirical inquiry of NIPT’s safety, clin- 
ical effectiveness, and cost-effectiveness. This phase involves both col- 
lection of quantifiable data and analysis of evidence. When the empirical 
evidence base becomes sufficiently robust and comprehensive, then the 
issue about its value is addressed separately: given these data, how is it 
possible to judge the overall value of NIPT? What are the ethical questions 
arising from the use of the technology? What is its social impact? In this 
perspective, values are viewed as separate from facts, and only enter the 
process when data collection is ended. 

Is this separation of facts from values attainable? Is this dichotomy 
desirable? Another way of proceeding is to start by wondering what it 
is expected by implementing this technology. What is it that one wants 
to realize, or avoid, by using NIPT? What is its relevance? From this per- 
spective, a group of consequences can be “plausibly” associated with 
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implementation and use of the technology. For example, one might want 
to prevent the birth of foetuses with chromosomal abnormalities. Neo- 
natal birth defects are a worldwide problem, with many countries having 
a high incidence rate of these defects. The extensive development of tra- 
ditional serological prenatal screening has made a significant contribution 
to reducing the birth of foetuses with chromosomal abnormalities, such 
as chromosomal aneuploidy. However, even though all high-risk pregnant 
women who receive traditional serological screening undergo invasive 
amniocentesis foetal karyotype analysis, there was still a high rate of mis- 
diagnosis in serological screening. NIPT can improve the detection rate 
of foetal chromosomal abnormalities and prevent the birth of foetuses 
with abnormalities. Another possibility is that one wants to avoid distress 
associated with an unexpectedly adverse outcome (for women and their 
partners). Because invasive tests have a small procedure-related miscar- 
riage risk, many women decline these tests, whilst those accepting testing 
experience considerable anxiety. Elevated levels of stress and anxiety dur- 
ing pregnancy are associated with potential adverse obstetrical outcomes 
such as preterm delivery and reduced birth weight and are therefore 
important to avoid. The fact of having a prenatal test for foetal abnormal- 
ities can affect maternal anxiety. A further plausible consequence is that 
one wants to facilitate reproductive autonomy. Reproductive autonomy 
is having the power to decide and control contraceptive use, pregnancy, 
and childbearing. For example, women with reproductive autonomy can 
control whether and when to become pregnant, whether and when to use 
contraception, which method to use, and whether and when to continue 
a pregnancy. NIPT allows to obtain valuable information on the health of 
the foetus. For pregnant women or couples, implementation and use of 
NIPT can offer reproductive autonomy regarding an affected pregnancy, 
including termination of pregnancy or being able to prepare for the birth 
of an affected child, relief from anxiety in case of a negative test result and 
the reduction of invasive follow-up tests. Finally, by using NIPT one might 
simply want to facilitate informed choice. Informed decisions are those 
that are founded upon relevant knowledge, and that enable a person to 
exercise autonomy. NIPT can facilitate a woman's reproductive choice by 
providing her with accurate information risk-free and early in her preg- 
nancy. 
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These different views (normative assumptions) do not only lead to con- 
flicts between stakeholders, but also between different components of 
an assessment. Furthermore, they play a key role in determining which 
facts are relevant and which facts are not relevant. For example, safety 
invokes, generically, the value of avoiding harm to people (the well-known 
moral norm of non-maleficence). What does it mean, exactly, “to avoid 
harm” to people in the case of NIPT? It should be clear that safety can be 
conceptualized in different ways, depending on the different normative 
assumptions (Bloemen et al, 2021). For example, it can be conceptualized 
in terms of avoidance of procedure-related miscarriages, or in terms of 
unnecessary worries due to false positive test results. In this sense, the 
assessment of the impact of a health technology in terms of safety always 
has a normative dimension. In a similar manner, clinical effectiveness 
refers to the commitment of doing good (the so-called principle of benefi- 
cence). Again, what does it mean to do good in this case? One possibility 
is to evaluate it in terms of predictive value. However, what constitutes an 
accurate and reliable test? Clearly, it is not a neutral exercise: acceptable 
thresholds of different components of test accuracy and reliability (e.g., 
sensitivity, specificity) invoke value judgments concerning the (un)desira- 
bility of certain outcomes, the acceptance of uncertainty, and the severity 
of conditions being tested for (Bloemen et al, 2021). Finally, similar consid- 
erations can be made about cost-effectiveness, whose outcomes can be 
expressed in different ways. For example, one possibility is to use Euro 
per quality-adjusted life years (QALYs) gained, another one is to use aver- 
age cost per trisomy 21 detected for different screening scenarios. 

This analysis shows clearly that the assessment of the impact of a health 
technology in terms of safety, clinical effectiveness, cost-effectiveness, 
always has a normative dimension. 
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CHAPTER 3 


Interpretive 
frames, judgement 
and action 


Abstract 


One central element of the VALIDATE approach is the recognition that an HTA needs 
to take into how the variety of actors involved in its object (caregivers, patients, 
policymakers, researchers, pharma-firms etc.) views on what is relevant, plausible, 
and valid with respect to the use of health technology. This chapter is focused on 
what is called in the policy sciences ‘interpretive frames’, a concept to systematically 


denote such ‘views’, 


It starts with a discussion of how interpretive frames help understand that different 
HTA researchers may adopt different outcome measures, and that these often do 
not take into account what outcomes are deemed important in clinical practice. 
Once the nature and role of interpretive frames have been pointed out, the chapter 
will discuss a method for systematically reconstructing and critically scrutinizing 
these frames and the way they are chosen. Finally, it will outline the ramifications 
of the often witnessed disjoint between HTA and clinical practice or policy making, 
and ways to pre-empt that disjoint. These various notions will be applied to the NIPT 


Case. 


Key message of this chapter: Different actors (doctors, policy makers, hospitals, 
patients, manufacturers et cetera) look into a health issue and associated health 
technology from different interpretive frames, and hence define problems and 
judge technologies different. Even within these professions different interpretive 
frames exist, and some may be underrepresented. Thus, first, reconstructing and 
scrutinizing interpretive frames may therefore help gain a more critical view on 
the object of the HTA, e.g., introducing a different comparator. Second, as different 
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actors prevail at the macro-, meso-, and micro-level, an HTA made at one level must 
take into account interpretive frames from the level that it seeks to inform. Under 
circumstances, this may have important methodical ramifications, toward forms of 
participative or deliberative HTA. 


After reading this chapter you should: 

= know the concept of interpretive frame and its rationale; 

= understand how interpretive frames shape outcome measures, comparators, 
and other key elements of an HTA; 

=" be able to systematically reconstruct and critically scrutinize interpretive frames 
and; 


= understand, on that basis, how to make an HTA optimally useful for clinical prac 
tice or policy making. 


Introduction 


What do HTA researchers do when they assess a health technology, or explore 
possible interventions to deal with a particular health issue? And why may they 
think that the resulting HTA will actually contribute to better practice? This chapter 
will discuss one theoretical notion, interpretive frames, and its implications for 
these two crucial - not to say: existential - questions for the HTA profession. Before 
outlining the argument more specifically, an example of a broader HTA project, in 
which interpretative frames played a key role, is discussed. The project was commis- 
sioned by the then Health Care Insurance Board (College voor Zorgverzekeringen 
(HCIB); now called Zorginstituut Nederland (ZIN))'s HTA department, which is 
supposed to underpin governmental decisions on insurance package - further 
called ‘the national HTA body’. 


It must be stressed that the case (mebeverine for treating patients with irritable 
bowel syndrome - IBS) comes from a 2003 project that is part of HCIB (later ZIN)’s 
continuous learning and consequently improving their processes for health benefit 
package management in the Netherlands (Boer, 2018; Couwenbergh et al, 2013). 
The national HTA body uses in its assessment the four key criteria employed in 


coverage decision-making: effectiveness, cost effectiveness, necessity and feas 


bility (Zorginstituut Nederland, 2013) and which are further operationalized during 
several years (Zorginstituut Nederland, 2017; Vijgen et al, 2018; Kleinhout-Vliek et 
al, 2017). 
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A first lesson from the example was the realization that different actors involved 
- doctors, policymakers, hospitals, patients, manufacturers, etc. - and HTA re- 
searchers themselves each have their own perspectives (‘interpretive frames’ in 
our terminology) on a particular health technology and its (side-) effects and cost 
became a key point of departure. Thus, in HTA studies, ‘the’ problem should not be 
considered as given at the onset of an HTA study - what the problem ‘really’ is, is 
rather one of several questions to be answered. 


Second, a key point of departure became the idea that effective health benefit 
package management was to take into account the interpretive frames ofthe meso- 
level (care organizations, insurers and so on) and healthcare professionals in devel- 
oping package decisions, as it had become clear that these shape actual healthcare 
practices. While this became even more important due tothe Health Care Insurance 
Act of 2006, which ended governmental involvement in implementation, we also 


argued that the more principled background is a second notion from the policy 
sciences: interpretive frames shape implementation action and therefore must be 
anticipated in effective policy design. All this led, third, to changes in HTA practices 
at more attention to ‘scoping’ (see Chapter 4), more involvement of relevant actors 
at different moments in the HTA process and more recognition that this process 


may benefit from iteration loops, rather than following a strictly linear path - typical 
elements of the VALIDATE approach. 


Thus, while this historical case does not to show the current state of affairs of HTA 
in the Netherlands’ ZIN, it is helpful to demonstrate the rationale and nature of 
some key elements of the VALIDATE approach, especially those related to problem 


structuring, interpretive frames and effective policy advice. The case focused on 
mebeverine as a medicinal treatment as part of the treatment repertoire of patients 
with IBS (Moret-Hartman et al, 2007). In 1995, it had been assessed by the national 
HTA body in terms of whether it should remain covered, following a parliamen- 
tary resolution seeking to better control health expenditures. As a basis for its 


analyses, the national HTA body had developed four assessment criteria (Table 1). 
Assessing mebeverine in these terms, the HTA body concluded‘: a) that the efficacy 
of the drug had been assessed when mebeverine was initially registered; b) that 
convincing evidence on the drug's effectiveness was lacking, as meta-analysis 
showed considerable differences between Randomized Controlled Trials (RCTs); 


6 Our translation in Moret-Hartman et al (2007) 
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and c) that its therapeutic value is limited. Despite these findings, the national HTA 
body recommended continuation of coverage since it considered availability of at 
least one medicinal treatment desirable and, compared with other antispasmodics, 
mebeverine was considered to have the least side effects. 


Table 1 Coverage decision-making criteria. Source: Moret-Hartman et al 
(2007, Table 2), summarizing Ziekenfondsraad (1995) 


Criterion Description 


Efficacy Its pharmacological action results in a therapeutic effect in 
clinical research (therapeutic potential) 

Effectiveness Its use in clinical practice results in the aimed goal of the 
treatment 

Therapeutic value The sum of its relevant characteristics (effectiveness, toxicity, 
user-friendliness, etc.) qualifying for its position relative to 
alternative therapeutic interventions 

Efficiency A medicine is effective and the balance between therapeutic 

(cost-effectiveness) value and costs is favourable in comparison to other treat- 
ments 


Contrary to this advice, the Minister subsequently decided to discontinue reimburse- 


ment. When this decision was taken to court by the industry, the court annulled this 
decision, arguing that the industry and the Ministry agreed on the inconclusiveness 


of evidence on the drug's effectiveness while the reasons to continue its use were 
much more important than the Ministry's concerns about making the health insur- 
ance package more affordable. 


Against this background, when the national HTA body took up the issue, it decided 
not to attempt to resolve the effectiveness question through commissioning a study 
to conclusive evidence. Rather, it would draft a tender for research exploring 
whether non-medicinal treatments could be promoted, especially dietary advice. 
Anticipating that the pharmacy department of the HTA body would not accept 
a social science study, it decided not to go for that most obvious option. Instead, 


it commissioned a preliminary study that would elaborate alternatives and assess 
mebeverine's therapeutic value in comparison to such other interventions, and thus 
prepare a more comprehensive study. Eventually, a small consortium was selected 


which would also use the views of patients and general practitioners (GPs) to deter- 
mine a suitable outcome measure. 
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The preliminary study reported that while literature showed important differences 
on the issue of appropriate outcome measures, practice was much less ambiguous. 
In interviews with GPs, patient satisfaction appeared the common criterion, some- 
times accompanied ‘decreasing symptoms’. As one researcher put it: “there are a lot 
of viewpoints in the literature, but in practice doctors take patient judgement as their 
guide” (Moret-Hartman et al, 1995: p. 19). The desired outcomes most mentioned 
by patients were less abdominal pain and less swollen belly; sometimes, also more 
frequent defecation was mentioned. On basis of these findings the report proposed 
a combination of a global assessment of patient judgement, complemented with 
changes in patient symptoms. In addition, it yielded a proposal for standardized 
diet: fibres, as they were the only dietary intervention with unambiguously proven 
effectiveness, and could be encapsulated, securing compliance. Finally, the consor- 
tium proposed a main study on the efficacy of mebeverine with both fibres and 
a placebo as comparators. 


The HTA body eventually decided that such a study would make little sense, given 
that the preliminary study had left two issues unclear: a) the methodological prob- 
lems behind the ambiguities in previous trials on mebeverine; and b) what would 
be needed to prevent these problems in subsequent trials. So, ironically, we see, 
(i) differing views in literature on outcome measures; (ii) a persistent use of effi- 
cacy (for admission decision) or effectiveness (to decide on reimbursement) in RCTs; 
(iii) a deliberate choice by the HTA body, and initially also by the team it grants the 
preliminary study, to use instead ‘therapeutic value’ as outcome measure, which (iv) 
is, in line with clinical practice, operationalized as ‘global assessment’ but (v) never- 
theless eventually replaced by a return to efficacy as the criterion for a follow-on 
study. So, in a variety of HTA exercises, diverse outcome measures are being used, 
but never the one that appears commonly used in practice. 


Interestingly, after two decades many of the insights that surfaced during this 
project and our analysis have become common practice. In 2019, the journal of 
the Dutch Medical Society published a review article for practitioners on state-of- 
the-art insights on belly complaints (Claessen et al, 2019). Referring to Enck et al. 
(2016: p. 8) and informed by clinical practice, it recommended as a first step in treat- 
ment explanation, reassurance, dietary and lifestyle advice; medicinal intervention 
may be a second step; when this is not (sufficiently) effective and the treatment 
relationship works well, behavioural, and psychological therapy may be considered. 


Citing recent research, they point out that hypnotherapy and cognitive behavioural 


therapy may be effective in mitigating bodily and quality-of-life symptoms (this has 
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been proven in clinical trials, but it is not clear whether this works for all patients) 
- they advise to refer motivated patients, who do not respond to treatment steps 
1 and 2, to a psychologist. Other developments in international literature are well 
in line with this. For instance, a review of qualitative research by Hakanson (2014) 
found that self-care management can be significant, and that it is shaped not also 
by the patient's physical condition, but also by her or his knowledge about disease/ 
illness-related matters, and her / his sense of agency, i.e., the awareness of the 
‘owner’ of one’s volitional actions. In addition, there is ongoing research on the brain 


gut axis which provides further evidence and insight on this dimension of pathology 
(see e.g., the review by Quigley, 2018). 


These developments, while partly showing progress in research, also reveal the 
‘wisdom of practice’ as it apparently developed, and thus raise a pertinent question: 
could a different approach to HTA not have helped to make patients benefit much 
earlier from that wisdom? That question yields additional depth to the above claim 
that this historical case study may serve to illustrate the rationale underlying the 
VALIDATE approach, and the emphasis in that approach of problem structuring, 


reconstructing, and evaluating interpretive frames and formulating effective policy 


advice. 


The case study suggests that asking the following questions will be helpful to explore 

this: 

1. Why is it that different HTA researchers may adopt different outcome 
measures? 

2. Why is it that the outcome measures used by HTA measures do not take into 
account what outcomes are deemed important in clinical practice? 

3. How may we critically assess outcome measures, and the way they are chosen, 
in HTA and in clinical practice? 

4. What may be the implications of the disjoint between HTA and clinical practice 
regarding outcome measures; and what could be the advantages of over- 


coming that disjoint? 


5. How could clinical practice be more taken into account in HTA? 


In the remainder of this chapter, we will answer these questions by introducing 
the notion of interpretive frames and explain how they inform what Donald Schón 
(1983) has aptly called reflection-in-action, i.e., judgement that informs action, not by 
preceding it but rather as interwoven with it, and embedded in the action context. 
On that basis, we will shed light on questions 1 and 2 above, by discussing how 
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different interpretive frames may prevail in HTA practices, medical literature and 
clinical practice. We will then explore how to appreciate these differences by asking 
the question how they may be critically scrutinized. Next, introducing an additional 
dimension of our case and drawing on some key insights from the policy sciences, 
we will argue for taking into account judgement in clinical practice in doing HTA and 
briefly indicate how this may be done. 


On action and interpretive frames 


When professionals do things, they cannot just be considered to be pre-pro- 
grammed automata, following if-then instructions. To be sure, under circumstances, 
they may operate rather instrumentally, constructing artefacts in a goal-following 
way - what Arendt has called ‘work’. Yet, the distinguishing feature of professionals i 
that they are able to deal with problems that need first be further processed befor 
goals can be assumed - involving ‘goal finding’ rather than ‘goal following’. Thi 
modus operandi Arendt has called ‘action’, a much more creative type of human 


(not necessarily merely professional) endeavour which essentially is driven by th 


S 
e 
ŞS 
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process of attributing meaning to a situation and to different ways of dealing with it 


nvestigating both from the perspective of one's wider beliefs and normative pref- 
erences. Action is driven by ‘reason’ rather than by ‘ratio’ (Toulmin, 2009), ‘reflec 
tion-in-action’ rather than ‘technical rationality’ (Schón, 1983) and ‘inquiry’ rather 
than ‘analysis’ (Dewey, 1938), to mention the language used by just some important 


authors have from the pragmatist tradition, in which the interwovenness of thinking 
and action is the central focus, as will be pointed out in Chapter 7. In this pragma- 


tist tradition, action and thinking are understood as essentially intertwined, and 
embedded in a particular context. 


Schón's depiction of these matters, what he calls an ‘epistemology of practice’, has 
been grounded in documented observations of professional action. What will be 
called in Chapter 5 - in the trail of Daniels (1979) - ‘the epistemology of ethics’, may 
be seen as a case (‘ethical practitioners’) in point. Schön conceives of action as a 
series of ‘moves.’ Each move is informed by the meaning attributed by the actor to 
the situation: what is at hand, and how do | appreciate that? That meaning consti- 


tutes the way in which the actor defines the problem; (s)he then attempts to move 


away from it. These moves are part of the actor's continuous reflection-in-action, 
or inquiry: they result from inquiry, in which the actor iterates between attributing 


meaning to the situation, considers different solutions and then decides on a first 
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move, reflecting a tentative view of problem-cum-solution; and the outcomes trig- 
gers further inquiry. Each move must be understood as a “conversation with the 
situation” in which “the situation talks back", and “the practitioner's effort to solve 


the reframed problem yields new discoveries which call for new reflection-in-action. 


The process spirals through stages of appreciation, action, and re-appreciation. 


The unique and uncertain situation comes to be understood through the attempt 


to change it... Furthermore, the practitioners' moves also produce unintended 
changes which give the situation new meanings. The situation talks back, the practi- 
tioner listens, and as (s)he appreciates what (s)he hears, (s)he reframes the situation 


once again” (Schón, 1983: p. 131-132) and acts on that. Precisely that constitutes the 
non- instrumental, creative nature of action, through which it differs from instru- 
mental work or routinized behaviour. 


Yet, this does not mean that action varies without limits, or into arbitrary direc 
tions. It is guided by the meaning the actor attributes to the situation, expressed in 
his problem definition. Attributing meaning is shaped by the lenses through which 
the actor interprets the situation, and although they may occasionally change in 
processes of ‘double loop learning’ induced by ‘crises and surprises’, these lenses 
are much more stable than the meanings, actions and assessment of solutions they 
give rise to, and which continuously evolve in day-to-day reflection-in-action. These 
lenses may be informed by education and are developed and reproduced through 
practice, where they yield patterns of action in particular kinds of situation. 


Elsewhere (Grin et al, 1996a) the notion of ‘interpretive frame’ has been proposed 
to denote the ensemble of a stakeholder's judgment of various solutions to a 
specific problem, problem definition, and the lenses that shape them. Following 
a scheme proposed by Frank Fischer (1980; 1995: p. 227-240), 2 x 2 layers may 


be distinguished. Two are specific for the action situation: the problem definition, 


reflecting the meaning an actor gives to the situation, and the actor's assessment 


of conceivable solutions (the actor's estimate of their costs, benefits, side effects, 


and risks). The other two are more generic, forming the lens through which the 
actor interprets specific situations: the actor's background theories (views and 


understandings of the processes and causal mechanisms playing a role in situa- 
tions like the current one, e.g., a doctor’s biomedical and physiological knowledge, 
or medical-technical insight) and normative preferences (including his sense of 
identity and preferred relations to others, e.g., his view on his relation to patients 
and their health condition). 
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Reconstructing interpretive frames 

In order to reconstruct an actor's interpretive frame, one may ask both direct and 

indirect questions to the either the actor (in an interview) or to written material 

(through a form of argumentation analysis). Direct questions, forthe respective layers 

ask: 

= what are costs, benefits, disadvantages, risks, associated with different 
proposed solutions? 

= what is the problem/challenge here, or: what is at hand and how do you 
appreciate that; or: what are key outcome measures/criteria for success? 

= what are key causal factors and mechanisms behind the phenomenon? 


= what is the actor's role/mission here, and what is the actor's identity, i.e., how 
does the actor see her/himself, how does (s)he) wish to relate to others? 


Indirect questions are ‘why questions’. As Fischer (1995: p. 230-233) has argued, 
each layer is connected to an adjacent layer by asking ‘why’ or ‘justification’ ques- 
tions. Asking ‘why is X a benefit?’ or ‘why do benefits X and Y outweigh disadvantages 
M, N? will yield (part of) the problem definition. Asking why particular outcome 
measures are important or asking why one is considered more crucial than another 
will shed light background theories. And identifying the strong and weak points of a 


specific background theory will mean to explore what types of problem definitions 


ad associate solutions it will produced, and what relations and identities these help 
reproduce. 


The methodological advantages of asking both direct and indirect question are 
twofold. First, this will yield more complete reconstructions of an interpretive 
frame: answers to different questions may complement each other. Second, it 
may yield more reliable reconstructions: if answers conflict, this is an alerting 
signal that one needs probe further, to remedy, for instance, a socially desirable 
or otherwise strategic answer to a direct question; if they are compatible, this is a 
valuable confirmation. 


Understanding differences in viewpoints 

We may now shed light on the first two questions above. First, different HTA 
researchers may adopt different outcome measures because they too differ in back- 
ground theories and preferences, and operate in different context. In our case study 
above the national HTA body's researchers, operating in a context where policy rele- 
vance is quintessential, defined the problem as the need to reduce the prescription 
of mebeverine while maintaining it in the package, and thus rejected effectiveness 
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as outcome measures and favoured therapeutic value; that also reflected their back- 
ground theories which included methodological knowledge and an understanding 
of placebo effects, as well as the idea that good policy solves the policy problem with 
measures that can actually be taken, and are based on an understanding of clinical 
practice. The team that did the preliminary study tendered by the HTA body to deal 
with that problem, on the other hand, operates in many contexts a year, and needs 
to maintain its independence and reputation throughout; it defined the problem 


as the need to find alternative treatments of which efficacy could be established, 
looking into the issue first and foremost through the lens of gastrointestinal back- 
ground theories. 


Second, for similar reasons GPs differ from both groups of HTA researchers. GPs 
define the problem of IBS as one which is primarily rather intangible (no effective 
medicinal treatment, counselling very time consuming, patients visit frequently) 
and thus tend to see patient satisfaction regarding the reduction of IBS complaints 
as the main outcome measure. This reflects their context as well as well as their 
medical insights (ambiguous aetiology, role of anxiety rather plausible). 


Critically scrutinizing interpretive frames 


Differences in context and interpretive frames explain why, and how, different actors 
each define the problem in their own way, pick associate outcome measures and 
look for solutions that make sense in these terms. But is this not mere relativism? 
As noted elsewhere (Grin et al, 2004), accepting that different actors in different 
context may define problem and outcome measures in different ways, this is not 
necessarily relativism because the recognition that there may be important truth 
in not just one, but a variety of claims, does not necessarily mean that one cannot 
tell that some claims are nonsense. One way to tell sense from non-sense will be 
discussed in Chapter 5: ‘coherentism’. in ways outlined in detail in there, essentially 


means testing whether “a proposition or belief is true and justified” by checking 
“whether if it is part of a network or set of ethical and other beliefs people hold that 
are jointly coherent”. 


In this section, a second, related approach, is outlined that may help to avoid 
relativism (and more fruitfully settle disagreements - (Banta et al, 1993: p. 152)): 
scrutinize claims on a specific situation and the background theories and prefer- 
ences underlying them and understand what the potential and limits of each of 


64 The VALIDATE handbook 


them. Importantly, as will be stipulated in Chapter 7, this could help HTA experts 
to play a significant, novel role: facilitate learning among the various stakeholders, 
generating new understandings and approaches to resolve problems. 


As Fischer (1995:, p. 237-239) points out, discussion of notions on these layers 
involves different kinds of argument. Discussing claims on the implications of solutions 
involves causal arguments, or empirical-analytical reasoning. In such kind of posi- 
tivist argument, it is possible to falsify claims when they violate facts, and to render 
them plausible by showing they are supported by facts. In our example, ironically, 
may be a placebo effect could have been most convincingly demonstrated. 


Scrutinizing a claim of “what the problem really is” or “what choice of outcome mea- 
sures is correct” essentially involves discussing what a situation means: that is, it 
involves a phenomenological argument (i.e., arguments rooted in our experience of 


a phenomenon). Now meanings are in the eye of the beholder, so here we cannot 
simply falsify meanings. Several things can be done, however, in order to critically 


scrutinize claims on ‘the’ problem or outcome measures. Such claims bring together 


(Hoppe, 2010) problematic conditions, i.e., observable features of the situation that 
co-constitute ‘the’ problem, and a problem definition, i.e., the way in which a specific 
actors sees the situation. So, one thing we may do to scrutinize an actor claim at 
this level, is to ask what observable problematic conditions support a particular 


problem definition. While this may not, in the same sense as for empirical-analytical 
argument, falsify a problem definition, it can help to assess its salience for the situ- 
ation. The other thing we may do is to compare the problem definition of an actor 


with the problem definitions or favoured solution of other actors, answering the 


question ‘in whose name may we define the problem in this particular way?’. In our 
example, the follow-on study proposed in the preliminary study was compatible 
with the typical interest of medical researchers (what about the efficacy of fibres) 
but largely neglected what IBS means for patients and GPs. 


To judge background theories, the key question is how they inform action. That 
involves hermeneuticinterpretive argument: for one or several situations, one 
considers what problem definitions and associate solutions would result when 
interpreting that situation on basis of this background theory. One test thus is to 
see, as just discussed, whose problem it emphasizes. In our example, this exercise 
would be salient, as it would point to the fact that the HTA project underemphasized 
the needs of patients and clinical practice. 
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Also, a more substantive test is possible, by asking what mechanisms and causal 


factors are uncovered when looking into the situation 


this background 


from the perspective of 
theory, and which ones are obscured. This, of course, requires a 


standard. For medical and clinical practices, one such standard has been proposed 


by Van der Wilt (1995), who distinguishes four determina 
tions, environmental factors and life circumstances, gene 
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Finally, by their nature, one cannot scrutinize normative preferences in terms of an 
external normative standard. Yet, as Fischer has argued and as Van der Wilt (1995) 
using cultural theory, Schwarz and colleagues (1990) and Thompson and colleagues 
(1990) have shown for health practices - normative preferences are dialectically 
related to different views on the determinants of care: they are mutually coherent. 
Scrutinizing these standards may thus be done by scrutinizing the ramifications of 
choices on the latter. 


Box 1 Underdeveloped background theories that matter: 

an example from the NIPT case 

When it comes to differences in empirical background theories that mat- 
ter in normative terms, there is an interesting parallel between the IBS 
and the NIPT case. In the IBS case, we have seen that already a decade 
ago there were ill-researched indications that alternative treatment 
might be effective enough to consider them more seriously when assess- 
ing mebeverine, such as dietary or psychological interventions. While 
that was recognized by GPs and other clinical practitioners, the fact that 
more formal knowledge was underdeveloped inhibited properly talking 
into account these options in the HTA study undertaken at the national 
HTA body. 

In the NIPT case, already authors like Diana Bianchi (2012) pointed out 
that, in principle, the field of genomics that was enabling prenatal tests 
like NIPT, could also yield novel opportunities for foetal therapy. Obvi- 
ously, the very existence of such opportunities might have a profound 
impact on the normative assessment of NIPT and could have given rise to 
a significantly different design and outcome of an HTA study. However, at 
the time, there was much less research devoted to the diagnostic use of 
genomics then to its application to treat, on basis of such diagnosis, fatal 
disorders through the application of personalized medicine. (Bianchi, 
2012: 1047-1049) This is probably one reason why the Dutch Health Coun- 
cil's assessment of NIPT (Chapter 1) briefly paid attention to prenatal ther- 
apy but did not include it into its conclusions. A second reason is that in 
the Netherlands, NIPT had been classified as a screening test, for which 
assessment criteria have been legally defined in the Population Screening 
Act; in that context, assessing NIPT in relation to other technologies was 
not appropriate. Thus, this HTA was shaped not only by different norma- 
tive viewpoints, but also by blind spots in mainstream medical research 
and by institutional context (Chapter 6). 
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Interestingly, and as another similarity to the IBS case, this blind spot got 
remedied over time. In more recent article, Bianchi and colleagues are 
pointing at interesting new research findings. For instance, different ways 
of scanning had shown that deviation from typical foetal development 
of the brain phenotype starts to occur by the second trimester, allow- 
ing time for intervention; and that mouse models suggest that medici- 
nal intervention with e.g., fluoxetine, continued for two years after birth, 
might yield improved brain growth and hence significantly reduce later 
cognitive impairment; and there are encouraging indications that this may 
also be the case in humans (De Wert et al, 2017: 223-224 and Hendriks et 
al, 2021). These articles claim that other possibilities for prenatal and ear- 
ly-age treatment are on the horizon. 

While the actual availability of such treatments would obviously lead to a 
very different problem definition for an HTA on NIPT, with different com- 
parators, and its outcomes would depend on a range of hitherto hardly 
explored considerations. For instance, there are still many issues that 
require empirical scrutiny (e.g., the degree to which reduced cognitive 
impairment also leads to psycho-social benefits and increased wellbeing 
- Hendriks et al, 2021: 2) and that may raise normative concerns (like the 
proposition that “people with DS [Down Syndrome] contribute to diver- 
sity, and that this is to be regarded as something valuable for society as a 
whole”. (De Wert et al, 2016: 224). 


In passing, the preceding discussion of how interpretive frames may be critically 
scrutinized has also answered another question: how such scrutiny may also 
contribute to improving scientific insight as well as clinical practice. This draws our 
attention to the relation between HTA and clinical practice. In the next section we 
will discuss this issue, building on the above discussion of interpretive frames. 


Implications of and solutions for the disjoint between 
HTA and clinical practice regarding outcome measures 


As we have seen, inthe IBS the precise definition of the problem differed significantly 
between HTA researchers in different context, GPs and patients. Furthermore, we 
have argued that this may be attributed to differences in context and the interpre- 
tive frames prevailing in those contexts. Thus, this disjoint is not an exclusive feature 
of this particular case, but it is plausible that it will occur much more frequently - it 
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has been documented indeed for a wide range of other cases, including the use of 
cochlear implants (Reuzel; 2001; Reuzel et al, 2007) and different ways of diagnosing 
and treating Parkinson's disease (Van der Wilt, 1995). 


There are three basic reasons why this disjoint is of concern. First, following 
Habermas (1981) a disjoint between the way in which a problem is dealt with in 
the system world (of e.g., informers of health decision-makers like the national HTA 
body and HTA researchers advising them) and how it is experienced in the lifeworld 
(of e.g., patients and the GPs, nurses and other first line practitioners they turn to) is 


normatively undesirable, because the power differentials between both worlds tend 
to de-privilege the lifeworld. How can healthcare system fulfil its basic mission at all, 
if it loses sight on the everyday life of the people it is supposed to serve? 


This admittedly rhetorical question has, of course, been recognized in the HTA 
community - it reflects the rationale underlying the emergence, over the past two 
decades, of patient involvement in HTA. As Facey (2017; p. 12) indicates, it is impor- 


nu 


tant to include patients' “experiences, preferences, perspectives” in HTA. Citing 
Coulter (2004), (s)he argues that this implies a need for greater patient and public 


participation, so as to ensure that the HTA focus on “the types of questions that 


patients want to be answered and engage(s) them in determining HTA priorities, 


designing and conducting assessments and appraisals, receiving and using findings 
from HTA and debating policy priorities and rationing”. 


Second, as we have seen, HTA research and, especially, medical literature on which 
it relies, may be limited by particular biases, emphasizing some determinants of 
a particular health problem more than others. This may affect the assessment of 
opportunities for more optimal care, explored by health practitioners but hitherto 
underexplored in literature. In the IBS case, the effectiveness of non-medicina 
interventions like diet had hardly been researched, hampering the opportunities to 


provide evidence-based advice. Similarly, while clinical practitioners are often wel 


aware of the influence of the brain-gut axis, here too lack of research on these issues 
led to a lack of proper evidence for intervention explored in practice. We have also 
seen that the understandings and approaches that had emerged as tacit knowledge 
in practice, were later confirmed and elaborated in more formal knowledge. Thus, 
in such cases, HTA researcher may wisely include recommendations to translating 
these experiences and insights from medical practice into directions of research 
programming to attain more formal knowledge on different understandings. 


Chapter 3 | Interpretive frames, judgement and action 69 


There is a dilemma here, however. As experienced HTA researchers are well 
aware, frequently the reverse issue is the case: pressure from - sometimes rather 
autonomous - clinical practitioners, hospital managers, industry and other health- 
care actors promote options that are not necessarily preferable from the viewpoint 
of societally optimal care (i.e., care that scores well in terms of the four coverage 
decision criteria from ZIN (2015): effectiveness, cost-effectiveness, necessity and 
feasibility). One important issue in such cases, like that ofthe Da Vinci surgical robot 
(Oortwijn et al, 2020), are the potential displacement effects: given constrained 
budgets, expensive novel technologies may draw resources from existing treat- 


ments or care provision (Wammes et al, 2020). Responsibly navigating between the 
two horns of this dilemma is a crucial challenge for scoping (see Chapter 4) as well 
as for ethical assessment (Chapter 5). 


Third, whatever intervention emerges from HTA work as advisable, it must be recog- 
nized that such an intervention may be done by a macro-level actor, but will only 
yield the expected outcomes through the efforts of actors at the meso (e.g., hospi- 
tals, manufacturers) and micro levels (e.g., practitioners, patients). Desired outcomes 
are matter of what Whitaker (1980) has called co-production between policy actors 


and actors involved in policy implementation. Co-production may occur if a policy 
that helps and incentivizes (Schneider et al, 1990) the latter actors to act in line with 
policy objectives. It is a key finding from implementation studies (Pressman et al, 1973; 


azmanian et al, 1989; Yanow, 2000) in the policy sciences and literature on the use 


O 


f knowledge in practice (Weiss, 1980; Dunn, 2008) that co-production is far from 
self-evident to occur. This can be well understood on basis ofthe above discussion of 


differences in context and interpretive frames (Grin et al, 1996a; 1996b). The crucial 
corollary is, that an intervention proposed in an HTA or a policy may be expected to 
have actual successif, and only if, patients, GPs and others consider actions expected 


fromthem sensible and fitting in their context their context. An action makes sense to 
these actors if it fits their interpretive frame - more specifically, if offers a solution to 
a problem that they themselves perceive and that does not violate their background 
theories and normative preferences (Grin et al, 1996b); evidence form a range of 
cases provided in Van de Graaf et al (1999). 


We have called this form of action-oriented agreement between actors differing 
in context and background theories congruency. Congruency is a less strict, more 
pragmatic form of agreement than the idea of having ‘shared objectives’, a ‘shared 
problem’ or even ‘shared values’, as it acknowledges that actors and their context 
may differ in nature. It is a more interesting form of agreement than ‘compromise’ 
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(e.g., between different groups of doctors, or between patient interest and policy 


objectives) as with a compromise all par 
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power position and strategic and tactical competences), while congruency entails a 


synthesis of viewpoints, which often 
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practice of most physicians. It would, h 


this process, as sta 


ted, that, at that tim 


ndardization can pre 


goes beyond a trade-off. 


ified as a congruent solution that, absent 


fying the best treatment for each patient 
e, “Evidently, this strategy is the current 
owever, be advantageous to standardize 


vent a significant amount of bias” (Moret 


Hartman et al, 2007). As discussed in the introduction, by now the formal knowledge 


has been developed that allows such standardization. 


Box 2 The disjoint between HTA and clinical practice regarding 
outcome measures in the NIPT case 

The three ramifications of the disjoint between the outcome measures 
used in conventional HTA studies and in clinical practice also express 
themselves in the NIPT case. First, the discrepancy between typical HTA 
‘system rationality’ and ‘lifeworld rationality’ expresses itself, for instance, 
in the use of the average cost per trisomy 21 detected for different screen- 
ing scenarios as an assessment standard (Bloemen et al, 2021) or to a 
“outcome measures such as cost per additional abnormality detected, or 
cost savings per disabled child not born, that frame NIPT as a technology 
that becomes more cost-effective when it prevents a sufficient number of 
births affected by genetic disability”. (Chapter 1) - standards reminding 
us of what Richardson (2000) once provokingly called the “Stupidity of the 
Cost-Benefit Standard”. The second ramification, the dilemma between 
giving too little or too much voice to clinical practice vis-a-vis formal med- 
ical knowledge might also express itself in the NIPT case. On the hand, 
it is clear from Box 1 that formal knowledge on pre-natal treatment of 
DS that may matter essentially for parents’ and society's appreciation of 
the NIPT test is as yet underdeveloped. The other side of the coin, also 
mentioned in Box 1, is that, should such treatments be available, there 
might be a risk that they are imposed by medical professionals or care 
institutions, despite potentially high costs and notwithstanding potential 
objections of denying the value of people with DS (cp. the reception of 
cochlear implants). 
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These two points together raise the question whether it would be pos- 
sible to come to a congruent outcome of the HTA, that would be norma- 
tively desirable from a societal standpoint (effective, cost-effective and 


yielding quality of life) and would be acceptable to parents: the third issue 
raised by the disjoint. 


For these various reasons, and as illustrated for the NIPT case in Box 2, it is 


important to ensure that the findings and proposed actions from an HTA study fit 


problem definitions for patients, doctors, and other key actors in clinical practice. 


Reaching an overall problem definition that represents such congruency requires 


proper scoping; doing so while navigating the dilemma sketched above may pose 


high demands on the art and craft of scoping, as will be discussed in Chapter 4. 


Iteratively finding solution(s) matching that specific problem definition (may be 


adapting the problem definition in the process) is a key task for subsequent activi- 


ties in the HTA processes, in which congruency may be designed through learning 
between the different actors involved (Grin et al, 1996a; Van der Wilt et al, 2015; 
see also Chapter 7). 


As Richardson (1990) has elegantly argued, this cannot be done by simply spec 


ifying norms or practice. Fundamentally, the reason is that practitioners are not 


rule-following beings. Rather, they define their actions in a much more reflective 


process of exercising judgement, as also portrayed by Schón. Judgement here refers 


to the Aristotelean notion of phronésis, which has been key to the development of 


interac 


tiv 


e Technology Assessment (Grin et al, 1996a; 1997). Phronésis for Aristotle 


is to exercise practical wisdom: knowing what needs to be done in concrete situa- 


tions in 


commi 


A key 


(0) 


tm 


rder to achieve maximal coherence among our multiple and varied value 
ents (Richardson, 1990; Fischer et al, 1993; 2012; Loeber, 2007). 


feature of phronésis being interpretation through a double hermeneutic 


(Yanow, 2000) of interpreting both generic principles and the particularities of the 


situatio 


n 


Following Arendt’s notion of “Im Stelle jedes Anderen denken” (think like, 


and for, all others), it essentially also involves that different viewpoints are being 


synthesized. Given its hermeneutic nature it may help understand the formation 


of interests, preferences, problem definitions and policy options. Obviously, such 


understanding is especially useful when analysing socio-technical change, in which 


preferences may evolve. Under circumstances stakeholders held different views 


on the technology; through frame reconstruction these could be resolved into 


different background theories; this occasioned partial revision (‘learning’), resulting 
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in agreement on howto proceed. Phronétic approaches to HTA may draw on recent 
extensions of HTA's methodical repertoire with more pragmatist approaches, such 
as methods for patient involvement as collaborative partners (De Wit et al, 2017), 
ethnographic field work (Tjrnhj-Thomsen et al, 2017) and, especially, deliberative 
methods (Street et al, 2017). 
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CHAPTER 4 


Scoping 


Abstract 


This chapter is focused on the art of crafting health technology assessment (HTA), 
i.e. scoping, and integrates the lessons of the previous two chapters. Because the 
relevance of facts depends on the adopted perspective, and there are different 
perspectives in society with respect to the desirability of health technology, the 
objectives of each specific HTA should be defined by taking into account these 
perspectives. Scoping concerns defining the objective and research questions of 
an HTA by a systematic exploration of relevant aspects from multiple perspectives 
(e.g. citizens, patients, informal carers, health professionals, decision-makers). There 
are several ways to do this, but it ought to be conducted in an iterative way, using 
explicit methods to elicit the values of different stakeholders, taking into account 
that these values may conflict. 


Key messages of this chapter: Scoping is the art of crafting HTA and is central to 
addressing relevant policy problems via meaningful assessments. Appropriate 
scoping, using explicit methods, result in findings and proposed actions from HTA 
that fit the problem representations of relevant stakeholders. 


After reading this chapter, you will be able to understand the importance of scoping 
as well as that scoping needs careful preparing, and the methods used to elicit 
stakeholder values are dependent on the context. 


Addressing relevant policy problems 


The type of policy problem addressed in health technology assessment (HTA) has 
an influence on the type of analytical approach that is appropriate, distinguished 
by the scope (program level or societal level) and focus (empirical and/or norma- 
tive). Translating the policy problem into research questions for each specific HTA is 
called scoping. More specifically, scoping concerns defining the objective and research 
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questions of an HTA by a systematic exploration of relevant aspects from multiple perspec 


tives (e.g., patients, informal carers, health professionals, decision-makers). 


Before describing scoping in more detail, we want to emphasize that - from its 


original intent - HTA aims to inform decision-making. This means that HTA is 


context-dependent and as such should reflect policy problems that are consid- 


ered important to society (see also Chapter 3), taking into account the complexity 


and dynamics of health systems (see 


also Chapter 6). However, the extent to which 


values at the societal (system) level are taken into account in HTA remains largely 


unexplored (Lóblová, 2018). 


HTA frameworks currently employed 


This is leading to insufficient sets of 


by HTA bodies are also not well-suited to take 


into account the wide range and diversity of stakeholder values (see also Chapter 2). 


relevant information. The assessment frame- 


works are typically based on contemporary HTA, using ‘substantive’ criteria, which 


are believed to reflect the most important social values. As such, decisions are often 


based on effectiveness (Wranik et a 
tions, using for example the cost per 


|, 2020) and in some cases economic evalua- 
quality-adjusted life years (QALY) approach. lt 


is known, however, that the QALY approach does not explicitly incorporate consid- 


erations of equity (Angelis et al, 2018). Lysdahl and colleagues (2016) also mention 


that “the majority of health economics guidance is based upon the assumption 


that assessment is seeking to support a global decision-maker engaged with maxi- 


mizing the efficiency of an overall health system....”, using cost-effectiveness thresh- 


olds. This focus may result in subopt 
many low and middle-income count 


imal outcomes at a societal level. Especially in 


ries, the use of cost-effectiveness thresholds 


could lead to flawed decisions on how to allocate scarce resources because deci- 


sion-makers do not consider maximi 
objective of the health system (Bertr 
that the ethical underpinning of cos 


zing efficiency to be the main or only value or 
am et al, 2016; Pichon et al, 2019). This means 
effectiveness analyses (e.g., maximizing effi- 


ciency) in themselves do not assure adequate ethical or legal reasonableness in 


coverage decision-making (Abrisham 


Itis important that the HTA communi 


relevant health system (societal level 
level), and the reasons why (Oortwijn 


i et al, 2017, see also Chapter 2). 


ty addresses more clearly what matters to the 
), as well as to relevant stakeholders (program 
et al, 2019). This is clearly reflected in the new 


definition of HTA (O'Rourke et al, 2020), and which is acknowledged to help HTA 


users to think beyond cost-containment, safety, and efficacy by addressing larger 


questions on the impact of health technology on sustainable ethical development 
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(e.g., in the case of COVID) (Mukherjee, 2021). Furthermore, practical guidance has 
been developed on how to link health system values to the HTA process, using 
so-called evidence-informed deliberative processes (Oortwijn et al, 2021). The guidance 
states that an HTA body or other relevant organization/committee which has this 
specific remit, uses relevant criteria for the assessment and subsequent appraisal 
of health technologies in line with the relevant health system values. 


An important issue to consider is, however, that different stakeholders such as 
patients, the public, providers, payers, industry, and policy makers, may have a wide 


range of social values and interests that result in different perceptions of which 
outcomes are considered to be desirable. For example, in decisions on public 
funding of expensive cancer drugs, patients may argue that the best treatment 
should be made available, other patients may argue that their treatment should not 


be displaced, and taxpayers may reason that it is important to make efficient use of 
public resources. As such, it might be challenging to find a common problem defini- 


tion to be addressed in HTA, as explicated by Gerhardus et al (2017) and in Chapter 


3 of this handbook. This challenge may pose high demands on the art and craft of 
scoping and highlights its importance in the HTA process: prior to conducting the 
assessment it should become clear what the problem is and what the related ques- 
tions are that the HTA should answer (Gerhardus et al, 2017). So, how to conduct 
scoping for a meaningful assessment that can be used to inform decision-making? 


Scoping the right research question(s) 


The explication and clarification of what makes a health technology desirable helps 
in identifying relevant outcome measures. This then provides important input 
for HTA, in the sense what evidence and other relevant information needs to be 
collected to answer the question(s). The available evidence can only be assessed 
for its relevance and completeness against the background of a specific definition 
of the policy problem (see Chapter 3). This may differ from health technology to 
health technology, but also from context to context. In Chapter 1, the case study of 
Non-Invasive Prenatal Testing (NIPT) was introduced, and it was emphasized that 
its purpose can be conceptualized in many, and sometimes contradictory, ways 
(see also Bloemen et al, 2021). 


Inthe Netherlands, for example, NIPT was assessed to determine whether 
it could contribute to improved prenatal screening by providing respective 
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parents with meaningful reproductive choices. This was defined as mak- 
ing a choice about severe health problems, in an informed way (respect- 
ing autonomy of the parents), and being proportional, i.e., respecting 
the anticipatory autonomy rights of the child (Health Council, 2013). 
In Germany, NIPT was defined in the HTA as a medical procedure (i.e., ability 
to save healthy foetuses from procedure-related miscarriages), and which 
was highly contested by different stakeholders in the country (Braun et al, 
2018; Bloemen et al, 2021). 


As such, consultation of stakeholders may be helpful in producing a variety of ways 
of how the problem may be conceptualized. 


If multiple problem definitions can be brought to light, the HTA team needs to 
resolve how this is going to affect the overall design of the HTA: which (elements of) 
of the problem definitions are going to be used to guide the HTA and what research 
questions will be addressed in an HTA, how they will be addressed, and to point out 
possible limitations. 


To produce HTAs that really address a relevant policy problem, it needs to be 
discussed and further defined prior to conducting the assessment. This is done 
through scoping. HTA agencies are often responsible for scoping, but policymakers 
(e.g., Ministry of Health), external committees, in consultation with relevant stake- 
holders and/or experts, can also do this. It appears that HTA agencies are more 
likely to prepare their own scope if they carry out their own assessment. 


Structuring the research question 


Most often HTA agencies define the research question(s) - via exploration of rele- 
vant assessment aspects - using the so-called PICO or TICO format. The PICO 
format is a system approaching used to describe which Patient, population or 
problem is targeted, which /ntervention is evaluated, which Comparator is used, and 
which are the relevant Outcome measures. The TICO format describes which Techno- 
logy is being evaluated, for which Indications (in terms target disease, population, 
and intended use), which Comparator is used, and which are the relevant Outcome 
measures. The PICO/TICO question may be extended by study design (S). In the 
context of complex health technologies, a more flexible approach may be required 
(Wahlster et al, 2016). 
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The choice of outcome measures relates closely to the criteria that are used in a 
specific context for decision-making. These decision criteria include, by default, 
generic criteria such as safety (i.e., avoiding harm), effectiveness (i.e., doing good) 
and quality of the evidence. In addition, these may also include a number of contex- 
tual criteria, i.e., those that are specific to the technology under evaluation. 


As stated before, stakeholders may have different views on the desirability of 
proposed outcomes. 


For example, in the case of NIPT reproductive autonomy could be a desir- 
able outcome considered from the perspective of pregnant woman in 
addition to “standard” health related outcomes, such as the QALY (Kessels 
et al, 2019; Bloemen et al, 2021). Furthermore, we already mentioned in 
Chapter 1 that outcome measures such as cost per additional abnormality 
detected, or cost savings per disabled child not born may conflict with the 
desirable purpose which is suggested by social and ethical analyses (Kibel 
et al, 2017). 


Furthermore, the way in which relevant outcome measures are conceptualized and 
operationalized is an important task in order to have a common understanding of 
its desirability. 


Bloemen et al (2021) concluded a normative analysis using an HTA report 
on introducing NIPT in the Netherlands. They found that safety was con- 
ceptualized in the HTA report as 1) the avoidance of procedure-related 
miscarriages (due to avoidance of using invasive tests) and 2) avoidance 
of unnecessary worries concerning the child’s health, due to false positive 
test results. The authors noted that it could be argued that other pos- 
sible consequences of taking a prenatal test should have been taken into 
account when assessing safety in terms of avoiding harm. For example, 
decisional regret, potential harmful effects on the foetus, societal pres- 
sure to take the test, and distress related to difficult decisions that need to 
be made as a consequence of test results, as well as societal views regard- 
ing people born with conditions after screening could have been con- 
sidered. This example makes is clear that the scope and outcome of the 
assessment is depending on how a criterion such as safety is conceptu- 
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alized, whose safety should be considered, and which (negative) impacts 
are important. Eventually, this may also influence conclusions regarding 
safety in the HTA report. 


The need to involve stakeholders 


In order to conduct a systematic exploration of relevant aspects from multiple 
perspectives, it is advised that HTA doers develop of an initial logic model to 
depict the health technology that is being assessed, identifying the relevant stake- 
holders involved and to structure the different outcomes that are found desirable. 
A logic model is “a graphic description of a system... designed to identify important 
elements and relationships within that system” (Rehfuess et al, 2018). A logic model 
can also help to take the variability of participants, context, implementation issues, 
and their interactions into account (see Chapter 6). The logic model can be used in 
scoping by iteratively involving stakeholders to identify (associate) outcomes, which 
may result in adapting the problem definition.’ 


From a survey among members of EUnetHTA is appears that in the scoping phase, 
industry, patient experts, clinical experts, payers, and providers are frequently 
involved. It is advised, that when determining the outcomes to be analysed for a 
specific HTA, to involve patients who are living with the condition in question, in 
order to ensure that the outcomes are important and relevant from a patient's point 
of view (e.g., fewer side effects, quality of life issues). Moreover, they may have infor- 
mation on the disease/condition and treatment process/options, which is not acces- 
sible to the assessors by evaluating clinical studies. 


If the patient is not able to communicate, as a result of the illness or 
because of being a child, a caregivers’ perspective (e.g., parents in the 
case of NIPT) may be useful. 


However, still then the question is whose perspective counts and why is it (not) 
considered in defining relevant outcome measures? Mercer et al (2020) explored 
experiences and perceptions among patient groups participating in the Canadian 
Agency for Drugs and Technologies in Health (CADTH)’s pan-Canadian Oncology 


7 See for a presentation on how to build a logic model in HTA (on reinforced models of palliative care): 
https://studylib.net/doc/9680608/presentation-on-building-a-logic model-of-integrate-hta. 
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Drug Review (pCODR) process. In the current appraisal process, the focus is on 
whether the clinical benefit of a new drug outweighs negative impacts (toxicity and 
associated side effects). However, patient representatives indicated that this was not 
a meaningful question as what is intolerable to one patient might well be tolerable 
to another. Thus, from a patient perspective, the more meaningful question “can the 
HTA process allow for differences between patients?” 


In the case of NIPT, the question could be asked whose health benefits 
should be considered: those of the unborn child of those of the prospec- 
tive parents? (Goldhaber-Fiebert et al, 2015). 


In addition to actively being involved, patient experts can also be asked to review 
scoping documents or attend (clinical) expert workshops. Clinical experts may 
be involved by providing data or being consulted on questions such as relevant 
comparator, clinical value of the product, how the product is used in clinical prac 
tice, implication in terms of resource utilization, etc. The role of providers in scoping 


is to provide data and evidence and they can be involved in stakeholder groups. 
Providers can review documents and provide general consultation via peer review. 
Finally, payers could also provide relevant information, data, and evidence in the 


scoping phase (EUnetHTA, 2017), but there may be other experts depending on the 


topic under assessment, 


Of course, one need to be aware that users/requesters of the HTA may be reluctant 
to include other perspectives than go beyond the area of direct (policy) interest. 
However, the involvement of relevant stakeholders to identify, reflect, and learn 
about the meaning and importance of relevant values and questions, and an 
evidence-informed evaluation of the identified values (criteria) can contribute to 
the legitimacy of recommendations and/or decisions, e.g., by improving the quality, 


consistency and transparency ofthe HTA process. This has implications for how HTA 
is conducted. 


Stakeholder involvement methods 


There are different levels of stakeholder involvement, ranging from communica- 
tion, consultation to participation, and it can be arranged using qualitative and/ 
or quantitative methods. Communication consists of only informing stakeholders, 
for example through public meetings, by dissemination to high priority groups 
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using patients’ organizations or by using social media. Consultation refers to a 
structured process for collecting feedback from stakeholders without providing 
opportunities for meaningful participation (i.e., deliberation). This can be done via 
several methods, including surveys, interviews, stakeholder meetings and solicited 
feedback from stakeholders (by invitation), or multimedia analysis (Abelson et al, 
2018). Stakeholder participation means that stakeholders are actively engaged in 
deliberations and can openly exchange views on argumentation and evidence, for 
example via nominal group techniques, consensus building approaches and expert 
elicitation techniques (Peel et al, 2018). In current practice, most HTA bodies apply 
communication with or consultation of stakeholders in their activities. 


Examples of methods for how relevant stakeholders could scope the HTA were 
explored in the INTEGRATE-HTA project (Wahlster et al, 2016). The INTEGRATE-HTA 
project aimed to develop concepts and methods for a comprehensive, patient- 


centred, and integrated assessment of complex health technologies that considers 
effectiveness and economic, sociocultural, ethical, and legal issues, patient pref- 
erences and patient-specific moderators of treatment, as well as context and 
implementation issues. These concepts and methods were applied in an HTA that 
examined home based palliative care services with and without an element of 
caregiver support. 


The following methods for scoping were applied depending on the context (Brereton 

et al, 2017; and Box below): 

= Stakeholder consultation: Local coordinators in England, Norway and Poland 
adopted the UK's philosophy for lay stakeholder involvement and all stake- 
holders were consulted as ‘research advisors’ to inform researchers’ decision- 
making in the project. Consultations were guided by the National Institute 


for Health and Care Excellence (NICE) methods for developing public health 
guidance (NICE, 2020; NICE, 2013) and the INVOLVE? briefing notes for involving 
the public in research. Information was collected and summarized using the 
EUnetHTA Core Model as an overarching framework for conducting the HTA 
(see Chapter 5 for more information on the EUnetHTA Core Model). 

= Qualitative research: A variety of qualitative approaches were used in 
Germany, Italy, the Netherlands, and Lithuania according to local tradition and 


Q 


8 AUK national advisory group, set up to support public involvement (i.e. patients, carers and those 
using health and social services) in National Health Service (NHS), public health and social care 
research. 
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researcher preference about stakeholder involvement. These included nominal 
group technique and categorical coding procedure following grounded theory 
methodology by Strauss et al (1990); interactive evaluation (see Chapter 5 

for more information) and subsequently case reconstruction using constant 
comparison and thematic analysis. 


In the INTEGRATE-HTA project, 132 stakeholders (80 professionals, includ- 
ing health and social care professionals/academics working in palliative 
care and 52 lay persons, including patients and families undergoing pal- 
liative care of 18 years and older) in seven countries (England, Germany, 
Italy, Lithuania, the Netherlands, Norway and Polana) highlighted similar 
issues affecting palliative care. This information complemented review 
level evidence and provided direction when scoping the HTA (Brereton et 
al, 2017). The contributions from lay persons primarily provided insights 
into patients' and carers' experiences of services whereas professionals 
were able to draw on their experiences of service provision to a wide range 
of clients and situations. This enabled the researchers to identify both an 
intervention and comparator model of service provision for the main HTA 
question. The issues raised were also used to inform sub questions for 
the assessment of specific aspects (e.g., ethical, socio-cultural aspects). 
In addition, some of the issues raised by stakeholders (i.e., the need to 
increase home care provision and for caregiver training/support), reso- 
nated with the findings of a review of review level evidence about models 
of service provision that had been completed at the same time. However, 
the study populations included no views of ethnic minority groups who 
are known to have specific palliative care need. As such, care is needed 
when interpreting the results of this study. 


Conclusion 


Scoping is a fundamental key element of HTA and the VALIDATE approach. The expli- 
cation and clarification of what makes a health technology desirable helps in identi- 
fying relevant outcome measures. This enables HTA doers and users to discuss how 
in the assessment certain facts can be collected. By relating the choices that are 
made in conducting the assessment and the, collectively determined, desirable 


ends of a health technology, the assessment becomes more meaningful for its 
intent: informing decision-making (Van der Wilt et al, 2017). 
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CHAPTER 5 


Ethics, ethics 
approaches in HTA, 
and ethics synthesis 


Abstract 


This chapter is focused on ethics and ethical analysis in HTA and starts with some 
meta-ethical reflections on how to view ethics from a linguistic, ontological, and 
epistemological perspective - presenting an alternative to a sceptic or nihilistic 
view of ethics. Then different methods for doing an ethics analysis in HTA are briefly 
presented, giving an overview of possible approaches that can be applied depending 
on the problem at hand. This section is divided into substantive, procedural, mixed, 
and interactive approaches and use NIPT to illustrate how the different approaches 
would handle a specific technology. Finally, the chapter ends with presenting how 
ethics input from existing literature can be synthesized. 


Key messages of this chapter: Ethical judgements can have a well-founded basis 
beyond mere attitudes and feelings. There are a number of approaches for doing 
ethics in HTA, and the choice of approach is dependent on both key values of the 


context and the type of decision-problem. In HTA, there are developed models for 


synthesizing literature on ethics. 
After reading this chapter, you will have a brief overview of meta-ethical standpoints. 


You will also understand there is a selection of ethics approaches to HTA and have 


a model for synthesizing ethics literature. 


Chapter 5 | Ethics, ethics approaches in HTA, and ethics synthesis 89 


Introduction 


The chapter starts with an introduction to the language of ethics and how this differs 
from other types of languages, i.e., what we do when we use ethical language. Here 
you can learn about what the foundation for the ethics (how we can say that some- 
thing is moral or not) and whether we can say that we know anything about ethics. 


Then we use this basis to investigate how ethical issues can be addressed in HTA. 


There are many ways to do so, and it is important to know which approach to apply. 
However, the choice of a particular approach is not value-free; it must be justified by 
a moral point of view (Patenaude et al, 2017). 


Lastly, you can learn about how to synthesize ethics literature. Here also there are 
several ways to provide an overview over the ethics literature on a specific health 
technology and to synthesize its insights. 


As described in Chapter 2, ethical issues ought to be an integrated part of the 
assessment of health technology. The extent to which these issues are explicitly 
identified and discussed may vary and will depend on the decision that is being 
informed by the health technology assessment (HTA), the expertise available, and 
the willingness to make ethics an explicit part of the discussion of the appropriate- 
ness of the health technology being assessed. At this point it is also important t 


O 


emphasize that the approaches require a certain degree of competence in ethics i 


5 


order to avoid a mechanistic use. 


(Meta)Ethics 


Is there more to ethics than just opinions? 

Often when people are discussing ethics, you might hear someone claim that 
“Well, there is no such thing as right or wrong, really! It is just a matter of opinion; it 
is all relative...!” (See also Chapter 2 on the right sink with (red) water from the well 
of emotions). Generally, people seem to base this idea on the fact that we disagree 
about ethical or value issues, more than we seem to disagree about issues that we 
do not consider ethical, what we might view as factual issues. 
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Within meta-ethics, the branch of philosophy dealing with issues about the ethics 
language, we find several different theories concerning how ethics should be under- 
stood. 


The ethics language 
When using ethical concepts like good, bad, right, wrong, brave, honest, sufferin 


ta 


etc. - we generally seem to be doing something other than just describing how the 
world is. When saying about a patient, that (s)he is suffering - it is not just estab- 
ishing a descriptive fact about the world. We also seem to imply that we should 
act on that information (e.g., to reduce the suffering if possible). Hence, it seems 


the ethics language is, in many cases, prescriptive. This prescriptive function of the 


ethics language is at the same time based on descriptive aspects of the situation. 
na more technical term, the value term, and its prescriptive function, supervenes on 
ore descriptive aspects of the situation. Hence, when we say about a professional, 


m 
that (s)he is acting wrong, we base this on empirical or descriptive aspects of how 
the professional behaves, what (s)he says etc. - e.g., hurting people, ignoring their 
decisions etc. Saying that (s)he is acting wrong is based on specific states of affairs. 


In some cases, when using ethics language, we are talking about how the world 
should be, rather than about how it is. Hence, when claiming about a specific tech- 
nology, that it is unethical due to causing patients harm - we are prescribing that 
the technology should not be used (regardless of whether it is actually used or not). 


In effect, the ethics language does, to some extent, have a different function from 


other forms of language. It is prescriptive and about how the world should be 
rather than about how it is. This brings us to the next questions: what are the basic 
elements of the ethics language, and how can we have knowledge in ethics? The first 
question is about the ontology of ethics and the latter about the epistemology of 


ethics. 


The ontology of ethics 

Now, even if we acknowledge that the ethics language is used with a somewhat 
different function in distinction to other types of language, itis still an open question 
what we are referring to when we use concepts like good, bad, right and wrong. Is 
there something “out there”, independent of us and our attitudes, we are referring 
to? Or are we simply referring to or even expressing our own attitudes towards a 


specific situation? 
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While religious traditions will refer to something outside us (God, Allah, etc), some 
secular traditions assume that ethics terms do not refer to anything outside us, 
but rather our attitudes in some ways. Hence, to claim about a specific technology 
that is it good - is to say that “I like this technology”. This kind of approach is called 
nihilism in meta-ethics and implies that when we use ethics terms, we are not saying 
something that has so called “truth value” about the world (but rather what has 
truth value about ourselves and our own attitudes) (cf. the discussion on scepticism 


in Chapter 7). 


To take a technology, where there is a strong controversy: euthanasia - i.e., actively 
to inject for example potassium chloride in a patient with the intention that the 
person should die. Some people find that euthanasia is right, and hence some- 
thing that should be offered to patients (under certain conditions). Other people 
find that euthanasia is wrong and should hence not be offered. For a nihilist such 
a discrepancy would simply imply that different people have different attitudes 
towards euthanasia (which in turn might depend on a number of different factors 


ike upbringing, culture, religion etc.). In essence, this implies that there is no real 


contradiction or conflict to resolve - we simply have different attitudes (like some 
might like coffee and others like tea). 


In the meta ethics discussion, perhaps the two most dominating arguments for 
nihilism is: 1) that there seems to be wide-spread disagreement between people on 
ethical issues; 2) that it seems difficult to resolve these differences through rational 
methods. However, it is not obvious that the disagreement is greater in ethical issues, 
than in other issues. Other issues are also difficult to resolve without questioning 
that such issues have “truth value” or exist independent of our own attitudes. To 


take a current example, even accepting the fact that humans’ impact on the climate, 


the best experts disagree about how much the temperature will rise within, say, 50 
years. Moreover, this seems impossible to fully resolve with rational methods. Still, 
we do think there will be a definite answer to how much the temperature has risen 
(or rather what the mean temperature is) in the world 50 years from now. So, the 


opponents of nihilism ask, is there really a principled difference? Why would such 
disagreement and lack of methods to resolve such disagreement have us conclude 
there is not truth value to ethical claims, whilst there is truth value to climate claims? 


The opponent of nihilism might instead turn to realism. A realist claims that concepts 
like good, bad etc. refers to aspects of the world, existing independent of man and 
our attitudes. Hence, if we claim that euthanasia is wrong, we do not (only) claim 
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something about ourselves, but about the world “out there”. Expressed in other 
words, the wrongness is a feature of the world and, hence, if someone claims that 
euthanasia is right, we are using contradictory concepts about the same world, 
which cannot at the same time both be true. Hence, for a realist, in such a situation 
we are having a real conflict or contradiction about the issue. So, what are the argu- 
ments in favour of the realist view about ethics? 


An indirect argument seemingly showing that we are not nihilists, but rather realists, 
is that we seem to treat ethical disagreements like real conflicts. That is, we try to 
convince opponents in an ethical debate that they are wrong, by using arguments 
to support our position, or to undermine our opponents’ position. In doing so, we 
use arguments we assume or hope the opponent will find convincing, i.e., we seem 
to assume we share a common value foundation of sorts. Hence, in arguing about 


euthanasia we might use arguments in terms of reducing suffering, respecting 
autonomy (of patients and professionals), causing harm, etc. Now, this might not 
necessarily show that we accept realism, we might try to convince only as part of 
promoting out own attitudes (without thinking we are right in a realistic sense). 


An argument following a similar line of thought is that assuming realism to be true is 
necessary to claim that someone made a mistake or got it wrong about ethics, e.g., 


in claiming that people in keeping slaves or viewing other ethnic groups as less valu- 
able did wrong - not only according to the standards of our time or of ourselves but 
from a more objective perspective. In effect, the realist claims that we seem to take 
ethics too serious to only be about attitudes, and moreover, seem to use similar 
methods to resolve disagreement as we use in other type of conflicts where we do 
assume there is an independent answer. 


A third line of argument for realism is that there seem to be certain issues where 
most people agree, e.g., with respect for vicious killing or torture. Hence, at least 
some issues seem to be handled as ethically absolute. 


The epistemology of ethics 

Even if we arrive at a realistic view on ethics, can we have knowledge about the values 
and norms we think exist? Or, to put it in other words, what does it imply to have 
knowledge in ethics. Traditionally, we define knowledge, as having true and justified 
beliefs about something. In meta-ethics, we find basically, two different theories 
about how to have true, justified beliefs in ethics: foundationalism and coherentism. 
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According to foundationalism, we should identify self-evident proposals about values 
and norms, which we cannot doubt as true and justified. These will then function as 
a base or foundation, on which we can build (by logical inference) other beliefs. In 
this way, we will have a set of beliefs in ethics that are true and justified, in resting 
on a solid foundation. 


Two problems arise for this approach. First, can we find such a solid foundation? 
Second, even if we have such a foundation, can we infer other relevant beliefs from 
the foundation? Much of the recent discussion has focused on requirements for 
claiming that something is ‘self-evident’ advocating different theories concerning 
this. Still, there is no agreement, and even if we would find such self-evident beliefs, 
there are still problems in what we can infer from such beliefs. To take an example 
of an ethical proposition that might seem self-evidently true and justified to people 
‘We should not inflict (instrumentally) unnecessary suffering to innocent people’. 
In order to accept such a proposition as self-evident, it needs qualifications about 
what is instrumentally unnecessary (e.g., suffering that is not required in order to 
avoid greater suffering etc.), what is suffering, who is innocent etc. Still, assuming 
such qualifications have been made, and been accepted - what can be inferred from 
such a proposition? Can we infer that we should (positively) benefit people or that 
we should respect the autonomy of people, from such a proposition? Not without 
adding a number of assumptions, that we might not find self-evident - e.g., that 
suffering also implies absence of benefit, that inflicting suffering has the same 
ethical status as abstaining from acting so that absence of benefit occurs, that what 
is unnecessary suffering is up to the person to decide on etc. etc. Hence, critics 
will argue that for most ethical beliefs we hold, foundationalism does not seem to 
provide justification. 


An alternative route is coherentism, arguing that an ethical proposition or belief is 
true and justified if it is part of a network or set of ethical and other beliefs people 
hold that are jointly coherent. In this case we do not need to find a solid, self-evident 
foundation, but rather modify beliefs until they are coherent. One way to express 
this coherent set, is to require that the set is in reflective equilibrium. This epistemo- 
logical approach has also been used as a methodology in ethics, in trying to analyse 


and modify propositions about ethics in order to make them coherent in relation to 
other sets of beliefs etc. a person has. 


The fundamental problem with coherentism is that two (or more), radically different 
sets of beliefs (ethical and others) might be internally coherent - hence, implying 
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a certain degree of relativism. Generally, it seems coherentism is more common 
among moral philosophers and ethicists, at least based on a more pragmatic 
approach, since coherentism seems more 'useful' when assessing ethical proposi- 
tions and beliefs. Still, it is a matter of discussion, exactly what should enter into the 
sets of beliefs to assess for coherence. A common approach is the so called wide 
reflective equilibrium, which according to Norman Daniels implies: “a method that 
attempts to produce coherence in ordered triple sets of beliefs held by a particular 
person, namely: (a) a set of considered moral judgments, (b) a set of moral prin- 


ciples, and (c) a set of relevant (scientific and philosophical) background theories 
(Daniels, 1979). In distinction to a narrow reflective equilibrium, we try to find equi- 


librium, all things considered (e.g., by testing a number of different theories and 


principles to which fits best, not only try to find coherence between a specific prin- 


ciple and a number of cases). Coherence does not only imply logical coherence but 


might also imply whether we are psychologically prone to have certain beliefs or not. 
For example, attitudes might be logically possible but empirically or psychologically 
uncommon or difficult to hold. 


To take an example, we are considering whether to introduce a new technology, 
with a small probability of curing a dying patient, but with a rather high probability 
of actually shortening his/her life instead. To find the true and justified standpoint 
for whether this technology should be used or not, from a coherentist perspective, 
we should identify relevant principles (e.g., non-maleficence, beneficence, respect 


for autonomy), and relevant scientific and other facts or theories (e.g., about patient 
autonomy, the trajectory of the disease the patient is suffering from, the biomed- 


ical mechanisms of the technology, data from clinical studies of the technology 


etc.). Based on these different inputs, we should find a reflective equilibrium. For 


example, what is a relevant balance between risk of harm and potential benefit will 


be a tricky issue, but an issue where we will look at other previous decisions in 
similar situations, to what extent we have allowed patients to strike the balance 
(without restrictions or not) etc. Since we are not only requiring logical coherence, 
but a more ‘empirical’ coherence, there might not only be a single answer to what 
is the right balance. Moreover, a further challenge is that everything in our set of 
beliefs etc. can be ‘up for grabs’, ¡.e., we might need to reassess how we used to 
interpret our principles, in the light of new scientific theories or in the light of new 
technologies etc. 
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Concluding meta-ethics 

The intricate discussions within meta ethics show that there can be more to ethics 
than just opinion, and many ethicists or philosophers would argue that there is 
more to ethics than just opinion. This section has aimed to give you a short overview 
of some theories in meta ethics, trying to answer questions about ethical semantics, 
ontology, and epistemology. We do not require you to decide which theory you want 
to side with or find most relevant, just reflect on how these theories might chal- 
lenge the preconception about ethics you might have. For example, whether you 


find there is far-reaching relativism when it comes to ethics or if you presume there 
is alarge overlap or consensus in the values and norms adhered to. 


Suffice it here to say, that in a given healthcare system, we will need to have a more 
or less consistent approach to which technologies we should use and in deciding on 
which technologies to use, we will need to provide arguments to support different 
alternative options. Hence, from a more pragmatic perspective, we might find a 
combination of a realist and coherentist approach useful. 


How to make an ethical analysis - different frames and 
approaches 


Even when we have some of the meta-ethical issues settled, it is not clear how we 
should act in specific situations or how we should evaluate what is good and bad 
with a specific health technology. Deciding on what is good and bad (values) and 
right and wrong (norms) is a task of normative ethics. There are a wide range of 
approaches in normative ethics. Here we will only present some of the main posi- 
tions that are relevant to HTA. Common to all of them is that they provide perspec 


tives on how to decide what is good/bad and right/wrong. Corresponding to the 
foundational perspectives in meta-ethics, substantive theories in normative ethics 


think that there are basic norms, values, or principles that can help us decide on 
moral issues (on technologies). Corresponding to coherentist perspectives there are 
procedural approaches in ethics. Let us briefly present each of them. 


Substantive approaches in ethics 

The main positions in normative ethics are consequentialism and deontology. The 
first argues that the consequences of an action are what is morally important while 
the latter argues that we have moral obligations that are decided by other things 


than consequences. 
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Utilitarianism: a kind of consequentialism 

According to consequentialism we are to act so that the consequences are maxi- 
mized. Accordingly, we are to implement the technologies with the best conse- 
quences. However, what are the best consequences? There are many answers to 
that questions, but one common answer provided by utilitarians is that we should 
maximize the total utility. 


n HTA it us urgent to maximize safety, efficacy, effectiveness, and cost-effectiveness. 


Accordingly, utilitarianism is at the core of HTA's traditional analyses of outcomes. 
However, as pointed out in Chapter 2, the ethical nature of this integration may 
not be recognized and ethical aspects may not be explicitly addressed (Hofmann B, 
2001; Hofmann, 2005; Hofmann et al, 2014). Integrating equity concerns in cost- 
effectiveness analysis is one example (Hofmann et al, 2014). Moreover, the goals 


(consequences) that are assessed may be unclear or diverse. There is a difference 
ifone aims at increasing survival and enhancing human capabilities (Van der Wilt et 
al, 2017). 


Hence, it is important to notice that consequentialism is a kind of normative ethics 


(especially in terms of utilitarianism) which is in many ways integrated in the tradi- 
tional way of doing HTA. 


A utilitarian analysis of NIPT would focus on and compare outcomes of 
the use of NIPT to alternatives to using NIPT. Outcomes can be health out- 
comes (reduced mortality and morbidity), values like quality of life, human 
capabilities, or ability to make informed (reproductive) choice, people's 
experience of potential discrimination for people with Downs syndrome 
etc. The decisive question is whether society would have a higher balance 
of positive consequences over negative consequences with NIPT com- 
pared to alternatives? 


Deontology 

Deontology is a branch of ethics inferring right action from norms based on ration- 
ally justified duties. Such basic duties (imperatives) are to treat people equally and 
as ends in themselves. Although some technologies may have a great maximized 
utility, we still are reluctant to use them. For example, NIPT can be used (with whole 
genome sequencing) to screen foetuses in order to select only those foetuses that 
are predicted to contribute most to a future society. A series of deontological argu- 
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ments can be made against such use, even though it could maximize utility, e.g., that 
it does not show the proper respect for human dignity, but also for it, e.g., that it 
enhances reproductive autonomy. 


Deontology can be used in HTA in many ways. It can be used as a “pure deontolog- 
ical analysis” or it can be applied as deontological parts of eclectic approaches. For 
example, several of the questions in EUnetHTA Core Model's Ethical domain could 
be viewed as deontological in perspective (assessment elements F0002, F0008, 
F0009, FOO14 in the EUnetHTA Core Model). 


Principlism - The four principles approach 

Another approach called “principlism” is based on common morality that all rational 
moral agents are supposed to share. According to principlism we share four basic 
ethical principles that can be applied to solve moral problems: respect for autonomy, 
non-maleficence, beneficence and justice. Principlism has been widely used in HTA, 
for example to assess public access defibrillators (HIQA, 2014). 


The four principles have a prima facie nature, which means that the principles must 
be fulfilled unless it conflicts with an equal or stronger obligation. The principles 
constitute a basic framework, and they need to be specified and balanced (i.e., the 
practical activity becomes that of specifying how the principles are to be used in 
specific situations and balancing the principles with the other competing moral prin- 
ciples). 


Principlism has been a popular approach in normative ethics in healthcare because 
it is simple and feasible. Its simplicity lies in the application of a stable set of ethical 
themes and concepts. However, this simplicity also constitutes the major limitation 
of the approach: the risk of leaving out a series of values and perspectives. Further- 
more, it has been questioned whether the (in this case four) principles (and only 
these) are universal. 


A principlist approach to NIPT would look at potential benefits to parents 
(and children), assess to what extent it results in acceptable harm or not, 
how it supports respect for autonomy and also, how it should be viewed 
from a fairness or justice perspective. On the latter principle, it could be 
related the degree of need for NIPT, but also how it would affect resource 
use in the rest of the healthcare system if used or not used. 
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Specifying norms 

Another critique that has been levelled at principlism is that it oversimplifies how 
general ethical principles can be brought to bear on concrete cases. According to 
Henry Richardson, a gap exists between general ethical principles and our ethical 
judgements of concrete cases (Richardson HS, 1990; Richardson HS, 1997; Rich- 
ardson HS, 2018). He suggested that in order to bridge this gap, general ethical 
principles need to be specified. This means adding clauses to the ethical principles 
such as what, why, when, where, how, by whom, and to whom, something may or 
may not be done. Richardson also holds that people are usually committed to a host 
of general ethical principles (e.g., fairness, liberty, sincerity, avoiding harm), and that 
our judgements and actions in concrete situations are often guided by multiple of 
such principles. In such cases, it may happen that these principles appear to lead us 
in opposing directions. This is where the method of specification comes in. The task, 
according to Richardson, is to revise (individually or collaboratively) the initial specifi- 


cations of the general principles involved and to try to advance novel specifications, 


such that the conflict is resolved. Richardson's model is quite complex, comprising 
formal rules that should be observed in such a process. Also, he is quite explicit 


about the assumptions underlying the model: it assumes non-commensurability of 


ethical principles, it holds that no single ethical principle is absolute, and it endorses 


a concept of rationality as seeking maximal coherence between our commitments 


to general ethical principles on the one hand and our daily doings and beings on the 
other hand. Tom Beauchamp, one of the instigators of principlism, has espoused the 


method of specifying norms as a compelling extension of principlism (Beauchamp 
TL, 2007). Richardson has himself elaborated how the method of specifying norms 
may be relevant in the context of HTA (Richardson HS, 2016). A practical example 


may be found in (Van der Wilt et al, 2018), applying the method to the case of clinical 
trials of Novel Oral Anti-Coagulants. 


Applying the method to NIPT, it would ask which general ethical princi- 
ples seem to be guiding communal judgements of this technology and its 
associated practice, and, if they appear to lead us in opposing directions, 
whether this might be resolved by advancing alternative specifications 
that would lead to greater coherence. 
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Procedural approaches 

Other approaches in normative ethics are less based on maxims, norms, values, 
or principles but are procedural. They provide a procedure to find the right answer 
to whether and how to implement and use a health technology in a good manner. 
The first approach has already been mentioned. 


Wide Reflective Equilibrium (WRE) 

Wide Reflective Equilibrium (WRE) is a coherentist model of moral argumenta- 
tion. As pointed out before, coherentist approaches are opposed to foundational 
approaches, which assume that there are certain undisputable basic principles from 
which moral judgments can be derived. In a coherentist approach, no such assump- 
tion is made. Instead, the validity of a moral judgment depends on the coherence (or 
mutual support) among general moral principle, moral judgment, and background 
theory. The method has become more widely known since it was used and advo- 


cated by John Rawls in his Theory of Justice (Rawls, 1971), where it is argued that the 


concept of justice as fairness is superior to the utilitarian concept of justice. To do 
so, Rawls argues that justice as fairness coheres with our considered moral judg- 
ments and is independently supported by background theory concerning human 
behaviour, such as risk-aversiveness and mutual cooperation. The method has been 
further elaborated by Norman Daniels (see above; Daniels et al, 2016). Examples 


of assessments of health technology where the method of WRE (e.g., Reuzel et al, 
2001; Daniels et al, 2016) has been used include telemedicine-supported home care, 


genetic engineering, and home environments for adults with significant disability. 


Applying this to NIPT, would imply that we look at the principles and judge- 
ments made in the system, and one crucial aspect would be to analyze 
how other fetal diagnostic tools are viewed from an ethical perspective. 
As coherence is central, accepting other diagnostic tools would provide 
prima facie reason to also accept NIPT. 


Casuistry 

Another approach that has been used to address ethical issues in HTA and which has 
procedural characteristics is casuistry. With deep roots in ancient moral philosophy 
and modern anti-theoretical bioethics, casuistry uses practical cases with an undis- 
puted solution to solve the moral challenging situation or dilemma in hand. Oriented 
away from theory or principles and towards the particular, the procedure in casuistry 
starts by identifying the structure of the case, i.e., by describing the circumstances 
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(who, what, when, where, how, by what means) and the relevant maxims involved, 
e.g., “the morals of the story”. Then it compares the case with similar “paradigmatic” 
cases. Paradigmatic cases are those where a solution is found which is generally 
accepted. The comparison of cases should reveal the moral maxims at stake and the 


subsequent practical implications. 


In HTA, Casuistry can be at play informally, e.g., when referring to solved cases such 
as coverage decisions, but it can also be more formally applied (Reuzel et al, 1999). 
As with other approaches in applied ethics, Casuistry has its shortcomings, e.g., 


potential changes in the value base between past precedents and current cases, 
and it “suffers from the potential limitations of relying on subjective analogic argu- 
ments and intuitive judgment about a particular case”, 


Applying this to NIPT could imply to describe how using or not using NIPT 
would impact on the pregnant woman and her partner from different per- 
spectives, on the professionals and potentially also the child. A paradig- 
matic case could be the use of ultrasound for fetal diagnostic purposes. 
Since fetal diagnostic ultrasound is acceptable, one can use this to argue 
that so is NIPT. However, there is a problem with this analogy (case) as 
there may be morally relevant differences between NIPT and ultrasound. 
NIPT is not applied to assess the date of delivery or get a nice picture to 
put on the fridge, but mainly to assess any deformations. 


Discourse ethics 

The central thesis of discourse ethics is that there is a “force of the better argument” 
driving towards consensus on certain norms and giving universal validity to some 
presuppositions of a moral discourse. It is based on impartial judgment and on 
arriving at consensus among those who are affected. While discourse ethics rarely 
is used in HTA, it inspires consensus-oriented methods in deriving legitimacy of 
formulating rules for technology use. One example is the interactive, participatory 
HTA approach (iHTA, see below). 


Discourse ethics can be used through an “argumentative discourse” among the 


HTA experts and other stakeholders, where all (present and future) interests of 
each potential stakeholder are taken into account. Ethical assessments adopting 


discourse ethics will be performed in a bottom-up manner. Each stakeholders’ 
perspective would influence “argumentative discourse”, informing/(re)defining the 
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overall HTA process. Used this way, discourse ethics would be implemented in a 
coordinated or interactive manner. 


Applied NIPT, discourse ethics might not be suitable to assess the specific 
technology, but perhaps more as a method to agree on common princi- 
ples for how to relate to fetal diagnostic methods. 


Hence, there are many approaches in normative ethics in general that can be 
applied to address ethical aspects of health technologies as part of HTA. Moreover, 
there are also other (mixed or eclectic) approaches that have been developed more 
specifically to address the normative issues with health technologies. Below we will 
give a short outline of some of these approaches in order to give you an overview of 
the field. The space does not allow for in-depth elaborations. For further details you 
may find the references useful. 


Mixed approaches 

EUnetHTA Core model 

The HTA Core Model 3.00 has been developed in the course of the European 
network for health technology assessment Joint Action 2 (EUnetHTA JA2). It consists 
of nine domains, among which is the domain “ethical analysis”. The “ethical analysis” 
domain is divided into six topics; three of them (Beneficence/Non-maleficence; 
Autonomy; Justice and Equity) are directly related to the Principilist approach to 
bioethics (see paragraph on Principlism). The other three topics are respect for 
persons, legislation, and ethical consequences of the HTA. Each topic consists of 
two to four questions, adding up to nineteen assessment “issues”. Authors of HTAs 
are encouraged to start by gathering information on ethical issues using systematic 
literature searches, professional guidelines, and the stakeholder views. In a second 


phase, it is suggested that users choose from different methods that have been 
assembled by a working group of the International Network of Agencies for Health 


Technology Assessment (INAHTA). The choice of the method should depend on 
factors such as the type of technology, the role and authority of the HTA organiza- 
tion, the time, and resources available and the expertise with ethical analysis avail- 
able within the organization. For a comprehensive analysis, it is recommended that 
more than a single method is applied, and experts in ethical analyses are involved, 
in addition to other scientists and clinicians. 
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The Socratic (axiological) approach 

The Socratic approach is axiological as it tries to uncover and highlight the values, 
norms and ethical challenges that are relevant for the health intervention, the HTA 
process, as well as for the decision-making process. The Socratic approach consists 
of six steps and seven basic morally relevant questions, which are further speci- 
fied in thirty-three explanatory and guiding questions (Hofmann et al, 2014). The six 
steps are: 
1. Identify the intended purpose of the health technology and reveal the back- 
ground for the assessment; 

2. Identify involved persons, groups, and stakeholders (e.g., patients, relatives, 
professionals, industry, health policy makers); 


3. Identify relevant moral questions (from a list of questions, Table 1) and justify 


the selection; 


4. Perform literature search in accordance with the identified moral questions; 
5. Analyse and discuss the moral questions identified (in step 3) on the basis of: 
- Existing literature and 
- Hearings / statements of involved parties (or their representatives) or qual- 
itative studies (relevant qualitative studies should be included in the litera- 
ture search); 
6. Wrap up and summarize the process. 


In relation to NIPT the Socratic approach would add consulting with stake- 
holder, and here a crucial aspects would be to identify a broad range: dis- 
ability organizations, parents that have chosen to abort a disabled fetus, 
parents who have decided to keep a disabled child etc. 


The Triangular model 

The Triangular model, known as the “personalist model”, is rooted in the human 
person (body-soul unitotality) as reference-value in the reality, according to the 
Aristotelian-Thomistic view, 


For NIPT, the fact that such a diagnostic tool is often associated with abor- 
tion (even if it need not be so, since the information might be relevant also 
for parents who will not consider abortion) - is highly relevant to the prin- 
ciple of defense of human physical life being primary to principles about 
personal freedom etc. 
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Consequently, the human person is the aim and the source of the society. This 
approach includes factual, anthropological and ethical data in a “triangular” norma- 
tive reflection process. The three steps of ethical process are: 1. Data collection 
(knowledge level): an in-depth study of all factual data concerning the object of the 
analysis; 2. Ethical/anthropological analysis (justifying level) according the following 


principles/operating criteria: a. the defence of human physical life; b. the intercon- 


nection between personal freedom and responsibility; c. the therapeutic principle, 
according to which the human person has to be treated as a totality of body and 
soul; d. the principles of sociality and subsidiarity, for which public/private bodies 
are called to help all persons, namely when they are not able to fulfil their needs. The 
way by which these principles are utilized is similar to Principlism methodology, but 


with a relevant difference: they are organized in a hierarchic manner, 3. ethical eval- 
uation (assessment) or appraisal (normative) level, that should address and facilitate 
the practical choices. Triangular model/personalist approach is considered among 
“local approaches” in the HTA Core Model. 


Interactive approaches 
We hope that we have not lost the reader with the various approaches for addressing 
the ethics of health technologies in HTA. 


The approaches to ethics in HTA are well described in reports (Hofmann B, 2006; 
Anderson et al, 2005; Lysdahl et al, 2016) and articles (Hofmann et al, 2015; Assasi et 
al, 2014; Assasi et al, 2016). It may seem frustrating that there are so many methods 
(Droste et al, 2010). However, HTA agencies are different, the healthcare systems 
vary, and the health technologies are diverse. Hence, one method may not fit all 
purposes. Therefore, it is important to know about the various available approaches 
for doing ethics in HTA. 


With the danger of increasing the confusion of the reader, but with the intention of 
providing an overview, we now will turn to another source of reflecting on the ethical 
aspects of health technologies. So far, the approaches have been based on norma- 
tive ethics, ¡.e., stemming from ethics and moral philosophy. However, there are also 
other sources for addressing ethical issues with technology, e.g., from the social 
sciences. We will below give a teaser of some such approaches. Common to many 
of these is that they stem from the Science and Technology Studies (STS) where 
science and technology are understood as in reciprocal and continuous interaction 


with society. The question is not as much how technology works, but more how to 
make technology work. 
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Constructive technology assessment (CTA) 

Constructive Technology Assessment (CTA) wants to narrow the gap between inno- 
vation and assessment by takingthe socio-dynamic processes into account. The core 
ofthe approach is an assessed implementation of a technology in society in order to 
improve the robustness of decisions about technology and to learn about and avoid 
possible harmful impacts. CTA includes four stages. First, a ‘socio-technical’ map 
identifying the most relevant social actors involved. The second stage includes early 
and controlled experiments, through which unanticipated impacts can be identified. 
Third, a debate between the various actors involved is organized. Finally, a synthesis 
report is written aiming at letting societal aspects of innovation become additional 
design criteria. CTA has been adopted as an approach to technology assessment 
by public organizations. The CTA approach aims to provide a broad assessment at 
an early stage of technology development. Discussions between researchers, engi- 
neers, manufacturers and future users are used in the development and diffusion 
of a technology to improve its (potential) effectiveness. In this way, the approach can 
be seen as a truly integrative method. 


When applied to NIPT, the focus would be on the question how this tech- 
nology might be adapted in such a way, as to conform to a greater degree 
with prospective users’ needs and demands. The technology can be used, 
for instance, to detect gross fetal abnormalities only (Richardson, 2016; 
Hofmann et al, 2014; Lysdahl et al, 2016), but also extended to detect still 
other genetic abnormalities, the clinical significance of many of which is 
presently obscure. 


Social Shaping of Technology 

Social Shaping of Technology (SST) involves different stakeholders in a real discourse 
on the assessment and implementation of a technology. The goal is to reduce bias 
and improve the validity and applicability of the HTA (Clausen et al, 2004). The core 
advantages with this approach are the fruitful analytical perspective on of how tech- 
nology is implemented in social practice as well as the symmetry between tech- 
nology and society, not only addressing how technology influences society (as do the 


technological determinists) or solely attending to how technology is constructed in 
a social sphere (social construction of technology). One of the challenges is whether 
the approach is an assessment of a technology or whether it is more part of the 
construction of it. 
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Interactive Health Technology Assessment (iHTA) 
Interactive Health Technology Assessment (¡HTA) is a specific approach to HTA, 


involving stakeholders throughout the entire assessment process, i.e., people who 


may experience the consequences of the assessment are involved in defining the 


research question(s) to be addressed (scoping), in designing the assessment and in 


the collection and interpretation of the data. 


The term ‘interactive’ refers to an 
explicit objective of the HTA is tha 


reconstruct and critically appraise the frames that stakeholders use to interpret the 


problem and to judge solutions. | 


t stakeholders learn from each other. iHTA aims to 


t does so e.g., by semi-structured interviews with 
stakeholders. Philosophically, iHTA is an approach to HTA which accepts fallibilism 


without embracing scepticism, an 


(Reuzel et al, 2001). 


d which puts primacy on practice. As such, iHTA 
can be considered to be firmly rooted in pragmatism. iHTA has been used to eva 
uate a wide range of technologies, both within and outside the healthcare domain 


interaction among the various stakeholders: the 


In the case of NIPT, it would ask who would be affected by introducing 
NIPT into prenatal care, and in what way, and try to involve those parties 
in the HTA process, collaboratively discovering differences and common- 
alities in how each of them frames the problem. This, then, is used to 
define the questions that will be addressed in the HTA. Stakeholders are 
also involved in discussing the practical implications of the findings for 
how the technology will be put to use. Overall, a strong emphasis is put on 
mutual learning among stakeholders and making use of their experiential 


knowledge (Grin et al, 1996). 


Table 1 Overview of the different approaches for ethics in HTA presented 


in this chapter 


Approaches for ethical analysis 


Substantive approaches 
= Traditional approaches in moral 
philosophy 
- Consequentialism 
- Utilitarianism 
- Deontology (Duty based ethics) 
- Principlism 
- Specifying norms 


Procedural approaches 
= Coherence analysis 
- Wide Reflective Equilibrium 
= Casuistry 
= Discourse ethics 
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Approaches for ethical analysis 


Mixed approaches 

EUnetHTA Core model 

Socratic (axiological) approach 
Triangular method (deontology) 


Interactive approaches 
Constructive Technology Assessment 
Social Shaping of Technology 
Interactive, participatory HTA (¡HTA) 


How suitable are the various approaches for addressing ethical 
issues in HTA? 

As can be seen, there are a wide variety of approaches available. This poses the 
question of which method to use. The answer is: it depends. It depends on the 
issues laid out in Chapter 1. Table 2 is a summary of the various approaches and how 
suitable they are for assessing health technologies depending on factual contro- 
versies, normative controversies, complexity, and meta-ethical presumptions. 
Hence, no method is perfect for all purposes and the context is crucial for deciding 
on the approach. 


Table 2 Description and assessment of the various approaches for 
addressing ethical issues in HTA the table is based on (Lysdahl et al, 2016) 
and (Hofmann et al, 2015) 


Approach Factual Normative Addressing 
controversy controversy complexity 
(Plausibility) (Relevance) 

Utilitarianism ot well suited Well suited oderate 

Deontology oderate Well ot well suited 

Principlism ot well suited Moderate ot well suited 

Specifying norms ot well suited Well suited oderate 

Casuistry oderate Not well suited ot well suited 

Discourse ethics ot well suited Moderate ot well suited 

Wide Reflective ot well suited Well suited oderate 

Equilibrium (WRE) 

EUnetHTA Core oderate Moderate oderate 

odel 
The Socratic (axio- oderate Moderate oderate 
ogical) approach 
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Approach Factual Normative Addressing 


controversy controversy complexity 
(Plausibility) (Relevance) 
The Triangular oderate Well suited ot well suited 
model 
Constructive tech- oderate We ot well suited(?) 
nology assessment 
(CTA) 
Social Shaping of oderate Well suited ot well suited 
Technology 
Interactive Health oderate Well suited ot well suited 
Technology Assess- 
ment (¡HTA) 


Synthesizing ethics input 


When we have decided on our meta-ethical perspective (or not) and on the posi- 
tion in normative ethics (or not) or have chosen a specific approach for addressing 
ethical issues with the health technology, most often we need to gather and synthe- 
size ethical input (e.g., from literature). Again, there are many ways to do this. For 
the gathering of ethics input from the literature, see for example (Droste et al, 2010; 
McDougall, 2015). 


If you want to assess or make a good critical interpretive review McDougall has 
suggested the following six features: 
1. Answers a specific research question, which may have been refined and deter- 
mined during the literature review process, and 
2. Analyses the literature as a whole as well as analysing individual findings and 
arguments within that literature, and 


3. Does not utilize rigid quality assessment criteria, but comments within the 
review itself on quality issues, and 

4. Generates theory and puts forward an argument about the literature, and 

5. Captures all the key ideas in the existing literature that are relevant to the 
research question, and 


6. Records and reports the search strategy (McDougall, 2015). 
What you synthesize is also of great importance. There are methods for synthesizing 


reasons (Strech et al, 2012) and arguments (McCullough et al, 2004; McCullough et 
al, 2007). An overview of the various methods for synthesis of the ethics literature is 
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available (Mertz et al, 2016) and in 2021 the Preferred Reporting Items for System- 
atic Reviews and Meta-Analyses (PRISMA) for ETHICS will be published. 


Conclusion 


In this chapter we have introduced the language of ethics and how this differs from 
other types of languages, i.e., what we do when we use ethical language. Here we 
have discussed the foundation for the ethics (how we can say that something is 
moral or not). Here we identified two views, ¡.e., nihilism and realism. Then we asked 
whether we can have knowledge about ethical issues, and we investigated founda- 
tionalism and coherentism. 


Thereafter we use this basis to investigate how ethical issues can be addressed in 
HTA. There we used various approaches from normative ethics but also from the 
social sciences. We illustrated that are many approaches and tried to provide some 
means for assessing them. 


Lastly, we provided some references for synthesize ethics input, such as from the 
ethics literature. The overarching goal of this chapter has been to give you an over- 
view of the field and make you ready to assess and start applying specific methods 
for doing ethics in HTA. 
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CHAPTER 6 


Context matters 


Abstract 


This chapter shows the importance to consider health system context during the 
health technology assessment (HTA) process, including the perspective of hospital- 
based HTA (HB-HTA), and how it may influence final coverage decisions. Context 
differences rely on the type of priorities, epidemiological paradigm, financial 
resources, number of trained professionals, ability of managers, and empowered 
patients, and are also influenced by levels of healthcare (meta: global policies; 
macro: system; meso: hospital, other settings; micro: clinician/patient/caregiver). 
oreover, there are geographical and socioeconomic differences among countries 
regarding infrastructures (transportation and mobility) that affect certain solutions. 
Furthermore, there are other aspects that should be considered when discussing 


the inclusion, use or exclusion of a technology in the benefit package or services 
to be provided within a healthcare system, such as sociocultural heritage, religion, 
or socioeconomic/geographic distribution of the population to be targeted. All of 


them influence the overall value and the values from decision-makers, which should 
be considered in both cases when assessing a technology and in deciding for their 
coverage in a health system. 


Key messages of this chapter: Acknowledging context, where decisions on payment 
for a technology should be made, is a key factor for a successful assessment. 
Different health system contexts can have different characteristics, values and infor- 
mational requirements that should be considered when framing and performing 
the assessment. 


After reading this chapter you will be able to identify what are the differential 


elements/ factors of a context to consider when planning, scoping, and performing 
an assessment, 
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What does context mean? 


Professionals differ from system to system (e.g., number, knowledge, skills); patients 
are diverse (e.g., age, ethnicity, sex, comorbidities, social support, caregivers); soci 
ties differ on their cultural heritage, values, religion, approach to health and wealth; 
health systems differ on their structure, technologies and infrastructure available, 
funding schemes, decision-making processes, epidemiological paradigm, priorities 
and priority-setting methods, among others; nations differ on their gross domestic 
product, the type of political system, the socioeconomic status of their inhabitants, 
the geographical and geopolitical characteristics of the country (e.g., centralised or 
decentralised, federal or regional) and the distribution of budget used for the health 
and social systems. All those and some other factors can be considered context 


oO 
1 


(see Figure 1). To which extent those aspects influence the need, and the value of 
health technologies is something that will be discussed in the following paragraphs. 
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Figure 1 Context related factors that influence the use and value of a technology 
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So, what do we mean when we talk about context? If we consider the definition by 
Cambridge dictionary of English language, context refers to “the situation within 
which something exists or happens, and that can help explain it” (Cambridge 
Dictionary, 2021). The Integrate-HTA project considered the following aspects those 
to be considered when talking about context and the implementation of a tech- 
nology: setting, geographical characteristics, epidemiological characteristics, socio- 
economic, sociocultural, political, legal, and ethical (Pfadenhauer et al, 2016). 


Context and its specificities 


HTA was initially aimed to answer decision-makers questions related to the appro- 
priateness to cover/reimburse innovative and new technologies at macro level, e.g., 
by health authorities at a country level or by insurance companies in charge of a 


¡ej 


ig amount of population (Banta et al, 1997). In an international meeting and work- 
shop held in Europe someone from the audience asked whether small countries 
must have HTA units, or they should rely on evaluations or assessments coming 
from bigger organizations and/or countries; others were questioning if hospitals 
in a country should rely on the assessments done by the National/Regional agency 


due to their different characteristics and cultures, as well as if any hospital should 
have its own health technology assessment approach. This question that seems to 


be easy to solve if someone carefully read the definition of HTA promoted by HTAi 
and INAHTA (O'Rourke et al, 2020), it is of full actuality in the “context” of Europe and 
within the world. 


During the last 15 years, HTA has been devolved to meso level, e.g., hospitals/county 
councils (Sampietro-Colom et al, 2016). One of the questions that frequently is 
raised is why this devolution is happening and why hospital/local organizations do 
not use the results provided by HTA agencies/units. The answer to this question is, 
again, because of context matters. The questions to answer at national/regional or 
local/hospital are the same: What clinical and healthcare benefit is the new tech- 
nology bringing? How do these benefits compare with the current clinical practice? 
Do the costs incurred by these technologies worth the benefits obtained? What 
would be the impact in the organization of healthcare? Considering that economic 


health resources are finite, what investments am | losing if | bet for this technology? 
While the questions are the same, the approach taken to answer these questions at 
national/regional versus local/hospital differs. When trying to answer the mentioned 


questions, it is necessary to follow and to consider what makes HTA different from 
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other health services research disciplines. And the difference is that HTA contex- 
tualizes the analysis considering the characteristics where the decision is going to 
be taken. Therefore, depending on the context where the decision is going to be 
made, the type and source of the information used for the analysis will differ. 


Is the context so deterministic as to provoke differences when informing decisions 
about health and health technologies? If so, which is the “least common multiple” 
or minimum context to which we should refer in order to establish HTA activities? 
The latest is an intriguing question, considering that any difference that justify the 
need to include new or diverse evidence to that available by other means or that 
modifies any of the parameters of the PICO question determines a new contextual 
element that should be taken into consideration, because that would influence the 
analysis of the facts (see Chapter 5). 


The minimum context should be the one that provides the whole information (facts) 
required to closest determine the value of a technology within given circumstances 
and framework, nevertheless the relevance given to those facts could differ. Thus, 
the HTA doer should consider all the conditions around the technology and its use 
that could affect the final value within a particular context and anticipate them 
within a realistic framework and required timeframe for decision (Buxton, 1987). 


Actually, context-based HTA analysis should not be far from what has been called 
Value Based Healthcare movement, which is determining or at least, approaching to 
ng 
issues or problems within a determined context should be considered part of the 


what the final value could be in real life. Reducing the uncertainties or anticipat 


HTA remit, which is, informing decisions laying all the cards on the table or as many 
cards as possible. 


Macro-, Meso- and Micro-levels of decisions 


When discussing context, someone needs to bear in mind the levels of decision and 
how that relates to which value(s) could be considered in each of those. Those deci- 
sions that occur at a continent wide, national or regional level are so-called macro- 
level decisions, although those at the global level are so-called meta decisions (e.g., 
where the budget should be allocated to: industry, education, health or when global 


organizations propose a vaccination policy to countries). Decisions at a healthcare 
facility level are considered meso-level decisions and finally, those that are targeting 
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the individual healthcare professional level when discussing around the manage- 
ment of a single patient or a subgroup of patients are named micro-level decisions. 


Typically, macro-level decisions relate to reimbursement or inclusion/exclusion from 
benefit package and overall organization and structure of services. In some occa- 
sions, they are focused on introducing public health interventions (e.g., vaccination 
campaigns, population based screening programs, drinking water treatments). 


When considering Nations/Regions, the first contextual aspect could be related 
to the different priorities countries could have, or the different needs countries/ 
hospitals may have stemming from the epidemiological or social point of view, even 
in a homogeneous space such as Europe or inside the same Nation/Region. The 
second aspect concerns the characteristics of the healthcare systems, that is their 


organisational features (financial schemes, procurers, providers and their interac 


tions), their health professionals (including culture, level of skills), the patients and 
their circumstances (most patients do not have a single pathology and the use of a 
technology could affect the evolution of the others) and the society and its values, 
in which they are embedded. The third aspect focuses on the characteristics of 
the technology, and how it relates to other technologies used for the same indi- 
cation (not all countries or providers have access to the same technology/ies for a 


patient or group of patients with similar characteristics), its mechanism of action, 
the expertise requires, the need to use other technologies, such as diagnostics, 
prognostic tools, data analytics or algorithms. Finally, other factors to be taken 
into account are the legal frameworks, the cultural and social features and the 
increasing concerns around the environmental consequences of producing, trans- 
porting, using, recycling or waste landfill of health technologies, among others 
(Polisena et al, 2018). 


Furthermore, apart from those inherent features fully related to the context, there 
are other factors that also influence, and they are not directly linked to the context, 
but the choices that those making decisions put forward. For example, the selec 
tion of the value framework by which a technology will be judged. Interestingly, a 
paper of the Global Policy Forum of HTAi (Oortwijn et al, 2017) pointed out that 


there is not a single value assessment framework for drugs, medical devices, public 
health interventions or other type of health technologies in a given country. Differ- 
ences typically occur when assessing oncology drugs, health technologies to be 


used to manage rare diseases or when assessing complex interventions. This shows 
that the value judgements when determining the value among technologies differ 
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among countries and within countries. An example of this is described in Kleijnen et 
al, 2016. They show that decisions around the reimbursement of a single oncology 
drug differ among selected European countries, even though they use the same 
clinical effectiveness evidence to support the decision. In Box 1, the case of Non- 
Invasive Prenatal Testing (NIPT) is presented. 


Meso-level decisions, i.e., those taken at healthcare provider (hospital) level, usually 
take into consideration macro decisions, but more frequently take into account their 
own specific contextual characteristics. On many occasions, meso-level decisions 
could influence the success of macro- and micro-level decisions, because they 
determine the resources, the organization of care and the access to diagnostic 
tests, treatments and rehabilitation programs. 


Several contextual items impact the approach when assessing technologies at 
National/Regional and local/hospital level, which include: geographical scope, 
assessment priorities, type of information needed, timing of the assessment, level 


and type of stakeholder involvement, and deliberative processes. 


The information that HTA National/Regional agencies/units provide through their 
assessments are usually of avery high quality and very comprehensive. Nevertheless, 
itis not enough, neither appropriate to decide ifa specific health technology should 


be introduced in a hospital. The documents performed by the National/Regional 
agencies/units usually look at the geographical scope of the health authority asking 


for the assessment, i.e., country, region, where different types of hospitals co-exist. 
These documents cannot be used to properly assess if a specific hospital needs 


a new PET scanner, if it has to introduce a surgical robot, or if it has to invest in a 
hybrid operating theatre. These decisions, for a specific setting (i.e., hospital) should 
consider the idiosyncrasy of the hospital, including the available technologies, the 
capacity and capability and experience of their professionals, their learning curves, 
the characteristics of the patients and the characteristics and competition of the 
different healthcare areas in the hospital. Documents aimed to support a deci- 
sion-making process at macro-level regarding a specific technology are perceived 


by hospital professionals as far away of their daily clinical practice, and, therefore, 
usually have little (or none) impact in the final decision taken at the hospital level 
(McGregor, 2006). 


Priorities in the type of technologies to be assessed also differ at National/Regional 
level versus local/hospital level. At National/Regional level, technologies prioritized 
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for the assessment are those that will be of interest of all (or most) healthcare 


centres at the country/Regional level. For example, the Spanish HTA Network of HTA 


Agencies assesses those technologies nominated and prioritized by the Council of 


assessed should b 


which not always 
oreover, technol 


type and characte 


inistries of Health of the Regions (Consejo Interterritorial), and technologies to be 


eofinteresttothe majority of Regions (Varela et al, 2018). However, 


technologies to be assessed by hospitals should be those of interest to the hospital, 


are the same to the ones prioritized at National/Regional level. 
ogies prioritized among hospitals ofthe same Region may not be 


the same, because technology demands from professionals differ depending on the 


ristics ofthe hospitals (e.g., university hospital, reference hospital, 


monographic hospital). Usually, high cost technologies, or those with a potential 


high impact on th 


e health system, are prioritized by National/Regional HTA agen- 


cies/units. Nevertheless, the technologies prioritized by a hospital could be not so 


expensive, but could represent an investment effort for a department of a hospital 


or for the hospita 


itself. One study carried out in Denmark showed that 70% of the 


demanded and assessed technologies by hospitals had not been assessed by the 


national HTA agen 


cy (Kidholm et al, 2009). 


As mentioned above, the information included in an HTA document for a hospital 


decision-maker, should be the information that the decision-maker find relevant for 


taking the decision. Therefore, the assessment should take into account the contex- 


tual characteristics of the hospital when taking decisions related to innovative and 


new technologies, 


later on domains 


such as investment, disinvestment and inappropriate use (see 
and context). Moreover, if doers of an HTA want that its recom- 


mendations will be considered when hospital decision-makers evaluate the intro- 


duction of a new technology, then, they have to perceive the results from the HTA 


as appropriate an 


d relevant. For that reason, the HTA process undertaken should 


not only be rigorous, it should also be transparent and examine all the information 


that contributes to carry out a contextualized analysis for the hospital. For this, it is 


highly important to involve the health professionals who ask for the new technology 


during all the HTA process. Recommendations from HTA documents performed 


without the participation of health professionals who will use the technology are 


perceived as theoretical or far away from real clinical practice and not acceptable 


(McGregor, 2006). 


Another contextu 


al item to consider in decisions at hospital level is timing. The 


assessment information always should be provided in time to allow decision-makers 


to take the decision; this timing is usually shorter at hospital level than at national/ 
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regional level. Traditionally, the documents produced by National/Regional HTA 
agencies/units are very comprehensive and robust, requiring a lot of time (some- 
times one year) if they want to follow the quality standards required internationally 
in HTA documents (EUnetHTA, 2020) (to mention that established HTA agencies also 
have some HTA products that are shorter and produced quicker). At the hospital 
level, decisions on technologies are made constantly and it is therefore necessary 
to have the information available as soon as possible. For that reason, HTA metho- 
dological instruments that conjugate rigor with timing requirements, are used at the 
hospital level. The mini-HTA is one ofthe most used methods (AdHopHTA mini-HTA 
template; Sampietro-Colom et al, 2015). 


Finally, the deliberative process to make the final recommendation also differs at 
hospital level from National/Regional HTA agencies/units. National/regional HTA 
agencies/units use deliberative processes (e.g., appraisal) that usually involve 
different stakeholders. Among these stakeholders, health professionals not directly 
involved in the demand of the technology and patient representatives are included 


in the appraisal committees (as well as representatives from industry in some HTA 
agencies) (Bond et al, 2020). In the hospital the recommendations are made with the 


results from the HB-HTA document and by the assessment team that, as mentioned 
above, also includes the health professional that was asking for the technology 


(though this may vary across hospitals). In hospitals, the patient is generally not 


represented when discussing the results of the HB-HTA to make a final recommen- 
dation. 


HB-HTA answers questions related with the appropriateness, feasibility, sustaina- 
bility, economic viability and opportunity-costs of introducing a specific technology 
in a specific hospital. In that sense, the difference between an HTA performed 
at national/regional level and an HTA performed at the hospital, is similar to the 
difference between the efficacy and the effectiveness. In other words, the results 
obtained from a technology in “ideal” conditions of clinical practice (efficacy) versus 
the results obtained when the technology is used in every-day clinical practice 
(Sampietro-Colom et al, 2016). 


Summarizing, HB-HTA is performed when HTA is carried out to inform a decision 
regarding a new technology at the hospital level. It is considering the hospital 
priorities on technologies to be assessed, the timing to give the answer to deci- 
sion-makers, the type of information and data that these hospital decision-makers 
contemplate as relevant for them, and involving the health professionals who are 
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asking for the introduction of the new technology in the hospital from the start 
(Sampietro-Colom et al, 2016). See Box 2 for the case of Photodynamic Therapy 
(PDT) for the treatment of basocellular carcinoma (CBC) as an example. 


Finally, micro-level decisions concern the care pathway of a single patient or a sub- 
group of patients. Micro-level decisions define what will ultimately occur with the 
individual patient, a treatment can be reimbursed by a defined system, or provided 
by a hospital. but if prescription is not made by the professional and administered 
to the patient or the patient denies the treatment or the proposed management 
scheme, the technology won't be used on those that could have an indication. This 
means that it is not only a matter of facts and how they could be collected to make 
decisions, but how information is transmitted and in which context and the percep- 
tion or perceptions that the different decision-makers could have at the different 


levels of decision-making. Health professionals with a different profile, knowledge 
or experience can act differently, patients of different age, socioeconomic status 


or suffering from different conditions (acute vs chronic, single morbidity vs multi- 
morbidity) could also have diverse perspectives and values concerning their 
pathology and thus, may influence the uptake of the technology. As it has been 


described in previous chapters of the VALIDATE handbook, values cannot be sepa- 


rately analysed when collecting facts, and also differ from context to context, 
including the relevance that is given to the different facts. 


Internal versus external validity and how that relate 
to context 


HTA provides the pieces of evidence and information required to explain how a 
health technology or a group oftechnologies will workin a specific context. As stated 
above, the information provided relates with clinical, organizational, and economic 
aspects; and depending on the level of decision-making, the PICO question, as well 
as ethical, social and legal aspects. Regarding the clinical impact of a technology 
(or group of technologies), HTA informs, for example, how much the technology 
will modify the final health outcomes in a patient, group of patients or system. It 
is clear that this relates to two epidemiological concepts, that is: “internal validity” 
and “external validity”. Internal validity is “the extent to which a piece of evidence 


supports a claim about cause and effect, within the context of a particular study” 
and the level of trust, we could have around this evidence. “External validity” relates 
to how applicable the findings are to the real world or other contexts (e.g., different 
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hospitals/healthcare centres with different organization of care, professionals 
with different skills and cultures; or different healthcare systems with, e.g., varied 
financing schemes). 

It should be considered that the same technology could be used under different 
circumstances, within different management schemes or standards, diverse 


patients, by different health professionals or caregivers and in different places/ 
settings. Therefore, when framing the PICO question (see Chapter 4), to which 
extent 
(I), dif 


ways of measuring the same outcomes (O), requires context-based in 


that use refers to different patients (P), within diverse healthcare systems 


ferent standards of care (C) or considering different outcomes or different 
formation, 
evidence or analysis. At present, the intended and unintended consequences of 


implementing a health technology or group of technologies have been simplified 


in several HTA domains (see Figure 2). In view of that, most HTA analysis consider 


domains such as: safety, effectiveness, economic aspects, patients and social 


n within 


aspects, ethical, legal, and organizationa 
netHTA (EUn 
could be shared across 


aspects. The HTA collaboratio 


Europe, EU etHTA,2021), considered that certain assessment results 


jurisdictions, such as safety, (relative) effectiveness, health 


problem, current use of the technology and the description of the technology and 


its characteristics. However, this is not fu 


safety or effectiveness could have different 


y true, considering that aspects such as 
results depending on the characteristics 


of the patients, or the systems in which those patients will receive healthcare. The 


former is exemplified by the differences 
populations for specific type of diseases 


in epidemiological characteristics across 


. For example, it is well known that the 


population from Southern Europe has bet 


ter cardiovascular profiles than the popu- 


lation from Northern Europe. This has been shown to impact the cost-effectiveness 
of a technology aimed to decrease high blood pressure in patients that were non 
respondent to drug therapy. While the cost-effectiveness analysis was positive for 
patients from Northern Europe (Borisenko et al, 2014) it was not when applying the 
same methodology to Southern European population data (Soto et al, 2016). The 
differences in the cost-effectiveness analysis (CEA) results were not just a matter 
of costs, it was mainly due to effectiveness results based on different epidemio- 
logical population profiles. It is obvious that including different population profiles 
and performing subgroup analysis (according to ethnic and geographical distribu- 
tions) could help, as differences can occur not only due to the genetic profile of 
the patients, but also from their lifestyle and environmental exposure; these could 
influence the final outcomes. Furthermore, it seems to be desirable to implement 
policies and methods that support and enhance the analysis of real-world data in 
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different contexts with the main purpose of analysing the value in real world condi- 
tions (values in this case are included per se). 


Another aspect that may impact the differences between internal and external 
validity is the relevance/importance that patients, caregivers, or systems could give 


to outcomes chosen in one place that may differ from another place. That said, value 
judgements on relevant outcomes cannot be separated from values in each given 
context. For example, just considering facts, a new drug that could treat patients 
with diabetes showed in ideal conditions (randomized controlled trial) an improve- 
ment in the data of efficacy on hyperglycaemic episodes. Nevertheless, the analysis 


of subgroups showed that not all the patients react in the same way and there were 
subpopulations that obtain better results than others. Depending on the charac 
teristics of the patients in a certain context (P), the outcomes that are produced in 
real world conditions (O) and the standard of care in that specific healthcare system 


(I), an HTA analysis could advise against the reimbursement of the drug, despite the 
data supporting a possible improvement in efficacy. That relates to facts, but what 
about the selection of hyperglycaemic episodes as primary outcome of interest 
without considering other factors, other outcomes and discussing their relative 
relevance with patients and caregivers. Perhaps, when discussing with them, the 


outcome of choice could differ and if so, the relevance of the differences in making 
decisions. 


The principle of caution when considering context 


Contextualization can lead to a problem of inequity and justice due to providing 
access to a specific technology in a given region and meanwhile, considering the 
lack of provision of the same technology in the neighbouring region for a patient 
with the same characteristics. Thus, Someone who aims to untangle the issue of 
value measurement and context should balance which the consequences of consid- 
ering the context to its maximum expression could be and how to better inform and 
diminish the conflicts that some decisions could infer. An approach to the context 
and the analysis of complex interventions is the Context and Implementation of 
Complex Interventions (CICI) framework as guidance for the assessment of context 
when assessing complex interventions. The CICI guidance introduces an overar- 
ching framework of the interacting dimensions of context (including setting) and 
implementation. This framework comprises eight domains of context (i.e., setting, 
geographical, epidemiological, socio-cultural, socio-economic, ethical, legal and 
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political) and four domains of implementation (i.e., provider, organization and struc 
ture, funding and policy) including definitions and descriptions of each of these 
domains (Pfadenhauer et al, 2016). This approach, although valid, continues being 
mechanistic in some way and obviates the necessary preliminary discussion on 
which the values according to the context are. Moreover, the new definition of HTA is 
currently including domains referred to context that are lacking in the CICI approach, 
such as concepts of life cycle and environmental aspects (O'Rourke et al, 2020). 


SCOPE HTA Core Model DOMAINS 


1. Health problem and current use of technology 


Rapid REA 


3. Safety 


5. Costs and economic evaluation 


7. Organisational aspects 


Comprehensive/ Full HTA 
l 


9. Legal aspects 
m 


Figure2 HTA Core Model Domains — Available from: https://eunethta.eu/hta- 


core-model/. 


Domains, its information requirements, and context 


Among those domains within the EUnetHTA Core Model, we have already declared 
that, there are some, that are specifically proposed to be related to the context 
in which the technology will be applied. Moreover, there are technologies that are 
affected by some domains more than others, so there is also a technology/domain 
dependency. For example, learning curves, related skills, need of sufficient number 
of patients and need of other resources are part of the analysis that relate to 
medical devices or population-based interventions more than to drugs, in which the 
definition of the indication and the correct prescription are more crucial than those 
organizational aspects already mentioned. Moreover, the relevance/importance of 
domains to consider in an assessment as well as the type of information required by 
decision-maker to make an informed decision is different at macro- and meso-level. 
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As mentioned above, the content and results of an HTA addressed to a specific 


hospital, should 
the aim is that c 
any decision rela 
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HTA Core model HB-HTA Core model 


a ms 
DOMAIN 


D1: Health problem and 
current use 


Y relevant YI most important 


D2: Description and technical 


ei: Vv relevant v relevant 
characteristics 


D3: Clinical effectiveness Y relevant V// most important 


D4: Safety aspects Y relevant V// most important 


D5: Costs and economic 
evaluation 


Vv relevant 


D5.1: Societal point of view Y relevant 


D5.2: Hospital point of view V/// most important 


D6: Ethical aspects Y relevant Y relevant 
D7: Organizational aspects Y relevant 
D8: Social aspects v relevant v relevant 
D9: Legal aspects Y relevant v relevant 


D10: Political and strategic 


aspects 
B v relevant 


D5.1: Political aspects 


D5.2: Strategic aspects VJV most important 


Figure3 Differences in the information asked by National/Regional and 


hospital decision-makers 
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All this relates once again to facts, but facts are affected by values and perspectives 
(see Figure 4). 
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Figure4 Perspectives around the same facts — Concept and script: Iñaki Gutiérrez- 


Ibarluzea 


Let us give some examples in which context-based analysis of some of the domains 

will result in the recommendation of inclusion, exclusion or modification of a tech- 

nology or group of technologies. 

= A national healthcare system is aiming to improve the outcomes of a popula- 
tion-based screening program by including a technology that allows auto-sam- 
pling in those subpopulations that show a low ratio of participation and a high 
rate of prevalence of the disease in late stages. The analysis of the available 
technologies shows that the reliability of the technology, in terms of sensitivity 
and specificity is alike to that of standard sampling. Nevertheless, the analysis of 
experiences in similar contexts reveals that the level of increase in the ratio of 
participation is scarce and thus the return of investment is no less than doubtful. 


In a regional healthcare system, there is a need to consider a new medica 
device, proposed by health professionals for the treatment of a mental illness 


that is refractory to other existing treatments and requires the permanent 


implantation of an electrode behind the skull of the patient, and a replacement 
of the battery every two years. The analysis of safety and efficacy and the costs 


of the technology including cost-effectiveness are in favour of the technology 


and its use. The health professionals (neurosurgeons) have the needed skills to 
perform the procedure that are required and have pressured the medical direc 
tors to be a referral centre. However, in a qualitative analysis of the acceptability 
of the technology by patients of the region, they mentioned that they did not 
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want to afford the risks of the surgical procedure and the battery replacement. 
Finally, the technology is rejected. 

= In a hospital, health professionals have the skills to perform a new surgical 
procedure and have required the managers to include it in the service package 
ofthe hospital. The managers asked for an HTA, an analysis ofthe number of 
patients to treat per year, and a comparison against the standards defined in the 
literature to ensure safety and quality of care is provided. This analysis deter- 
mined that the number of patients to treat are far below those described in 
standards, thus the technology is not included. 


Finally, apart from the classical domains described, there are other domains that 
are gaining insight among those performing HTA. Recently, environmental analysis 
is being included in the new HTA definition among those domains to consider when 
providing evidence about health technologies (Polisena et al, 2018; O'Rourke et 
al, 2020). Those newly included domains apart from comprising new facts for the 
assessment include values that could influence the final decision. It is worth noting, 


the objectives of the millennium (United Nations, 2021) include good health and 
well-being, responsible consumption and production and climate action, among 
others. The consideration of those goals by countries could influence decisions 
around procurement, reimbursement, or inclusion/exclusion of health technologies 
in the healthcare systems. 


The life cycle of health technologies and context 


The new HTA definition (O'Rourke et al, 2020) also included another important 
change which is the life cycle concept, i.e., the assessment from its early develop- 
ment (Tummers et al, 2020) to reassessment when well established into a health 
system. Classically, HTA informed decisions have been related to inclusion of tech- 
nologies in benefit packages, reimbursement schemes and procurement processes. 
Nevertheless, increasingly HTA has been used to inform decisions around health 


technologies related variability in practice and decisions on disinvestment of health 
technologies (Ibargoyen-Roteta et al, 2010; Gutiérrez-Ibarluzea et al, 2017). The value 
of a technology should be considered within a context, in comparison to other tech- 
nologies in use for the same indication and within a time frame in which facts are 
obtained. New pieces of information provide new evidence around the technology 
in practice when applied to patients, the launch of new technologies can modify 


the line of treatment in which the technology can be included, and new data can 
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supportthe disuse of a technology. The value and the values are not immutable, and 


they are affected by new data, by societal/cultural changes or by the entrance of new 


technologies. HTA doers and decision-makers should be aware of that and promote 


reassessments of existing technologies that help reconsidering the management of 


single or subgroups of patients or modify decisions (Soril et al, 2018). 


The legislative context 


Finally, the characteristics of the legislative context may influence the final uptake of 


the health 


technology assessed. Health is a universal right that has been mentioned 


in international agreements which include the Universal Declaration of Human 
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Box 1 Context & macro-level decision-making: NIPT as an example 
of context-based coverage decision-making 

Nowadays, NIPT is heterogeneously covered across different health- 
care systems (Table 1). The approach to assess (analysis of facts) and to 
appraise (reaching a decision from the facts) this technology has inher- 
ently embedded values, and the mentioned heterogeneity is a result of 
the differences in values that prevail in different healthcare contexts. In 
other words, the values considered in both mentioned steps of the HTA 
process (i.e., assessment and appraisal), will determine the final target 
population who will have access to the technology as well as who is going 
to pay for it. As shown in Table 1, and with the same available scientific 
evidence, countries differ in who have access to NIPT. We can also think 
the other way around; do the established system, the financing schemes, 
the providers of services and access to the technology by patients influ- 
ence the analysis of the facts? Do the same aspects influence the deci- 
sion based on the same facts? Obviously, yes, in fact, the questions that 
are asked in different systems, although they could seem to be the same, 
interestingly differ and so the facts that will be considered and how they 
will be considered. 

The assessment (analysis of facts) procedure is approached differently by 
countries. For example, while the Health Council of the Netherlands has 
based its coverage decision mainly on effectiveness criteria and ethical 
and social issues (safety - i.e., avoidance of invasive prenatal test pro- 
cedures and procedure-related miscarriages-, reliability of test results, 
higher potential uptake, routinization, reproductive choices, persona- 
lized medicine); the Belgian Health Care Knowledge Center based its 
recommendations on a cost-effectiveness analysis looking at the aver- 
age cost per trisomy 21 detected for different scenarios (and not the cost 
per Quality-Adjusted Life Years (QALY) used frequently by HTA agencies) 
(Gadsboll et al, 2020). 

In the appraisal step, when coverage decisions are made (level of cov- 
erage and access to the technology), embedded contextual values also 
determine its results. While in some countries no access (non-coverage) to 
NIPT exists, others have implemented NIPT for all pregnant women. Most 
countries are implementing NIPT in a subset of pregnant women popula- 
tion, those who have intermediate-high risk for foetal aneuploidies after 
combined first trimester screening (cFTS), which is the gold-standard diag- 
nostic test for foetal aneuploidies nowadays. Nevertheless, also in this 
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target population, there are differences between health authorities when 
defining the criteria that include intermediate-high risk pregnant women. 
For example, in Canada, two provinces have different criteria for cover- 
ing NIPT with public funds in intermediate-high risk pregnant women. 
Ontario has a broader number of criteria (with two types of Category and 
a total of 10 criteria included) (Ontario Health Insurance Plan, 2020), while 
Vancouver (British Colombia) has defined only two specific indications 1) 
if a woman has had a prior pregnancy where the baby was diagnosed 
with Down Syndrome, Trisomy 18, or Trisomy 13; or 2) if a woman has 
received a “positive screen” based on a different prenatal screening tests 
(Quad, IPS, SIPS, FTS) (Pacific Center for Reproductive Medicine, 2020). 
Here, again, the inherent values of different healthcare contexts result in 
different decisions regarding access to NIPT. 
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Box 2 Context & meso-level decision-making (Hospital-based- 
Health Technology Assessment-HB-HTA): Photodynamic Therapy 
(PDT) for the treatment of basocellular carcinoma 

Hospital-based Health Technology Assessment (HB-HTA) is not defined by 
the setting where it is being carried out. In other words, because the HTA 
is performed at the hospital level it is defined by the perspective taken 
when carrying out the assessment, which is the perspective of the hos- 
pital where the decision about the introduction of a new or innovative 
technology is going to be made. The key question is to wonder about the 
implications and impact for a specific hospital of introducing a specific 
technology. To illustrate this concept, take Photodynamic Therapy (PDT) 
for the treatment of basocellular carcinoma (CBC) as an example (Van 
der Wilt et al, 2016). An HTA from a National/Regional perspective will 
assess whether this technology can decrease the burden of the disease, 
and whether the quality and efficiency of the care provided will improve 
when this technology is used instead of other available technologies for 
the same clinical indication. It will also assess which patients, or subgroup 
of patients, can benefit the most from this technology. For that purpose, a 
review and synthesis of the scientific evidence will be carried out consid- 
ering randomized controlled trials, cohort studies, case-control studies, 
and other study designs. Additionally, costs associated to its acquisition 
and use of the technology will be explored to include them in a cost-effec- 
tiveness analysis to elucidate if it is an efficient option. From the results 
obtained, recommendations regarding who, when and under what con- 
ditions should the PDT be adopted will be made; as well as the needed 
steps to appropriately endow economically the new technology to be 
introduced into the health system. Up to here, the HTA has demonstrated 
the added value of the PDT for CBC patients in average healthcare cen- 
tres. But HB-HTA goes further and will look at the potential value that this 
technology may reach in a specific hospital. First of all, it will be neces- 
sary to look at the effectiveness and cost section of the HTA and decide 
how much can be translated to the situation at the hospital (i.e., are the 
patients in the document similar to the ones of the hospital?, are the costs 
similar to the costs of the hospital?, etc.). In other words, firstly it will be 
necessary to review the transferability of the results to a specific hospital 
(Chase et al, 2009). Secondly, there will be a need to look for additional 
information and answer other type of questions such as: 
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= Is the hospital treating patients with CBC and which departments are 
involved? (e.g., dermatology, ear-nose-throat department, general 
surgery etc.) 

= The available treatments currently used for CBC patients: are they 
going to be completely replaced by the PDT? What will be the conse- 
quences for the organization of this type of care and for the health 
professionals? Will operating theatres be released? If this is the case, 
will operating theatres be used for other alternative surgeries? 

= Is it necessary to make investments (to buy the technology, to train/ 
educate professionals, to adapt the space for technology allocation)? 
Are the investments to be recovered? What is the return on invest- 
ment? Is there any reimbursement by payers? And how much will the 
reimbursement be? 

= Is there any other centre around that is already offering this treat- 
ment? Is this service aligned with the overall strategy of the hospital? 

= Will a qualified nurse be able to offer the PDT, under supervision of a 
dermatologist? Is that going to be legally accepted? Is there any risk or 
penalization associated to this situation? 

= Who is the manufacturer of the equipment? Are there several manu- 
facturers? Can we carry out a price negotiation for buying the equip- 
ment or training the professionals? 

= Should/can the equipment be integrated in the information system of 
the hospital? 

= Is the treatment offering opportunities for newer research or for 
training of residents? 

= Is there any probability that the incidence of CBC will increase in the 
near future? Is there any other pathology that can be also treated with 
the PDT? 

= Is there any other technology in development that will make PDT 
obsolete in a couple of years? 

= Is the PDT a treatment only to be used by university hospitals? Or is 
there any chance that in few years this treatment will be also offered 
at primary care level? 
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VALIDATE approach 


Abstract 


Because the VALIDATE approach is based upon the premise that normative analysis 
is, and should be, an integral part of HTA, it needs to address the possibilities and 
limitations of reaching agreement on normative issues in the context of health 
policy. Although a strict fact/value dichotomy is criticized (Chapter 1 and Chapter 2), 
and the plurality of methods for conducting normative analysis is recognized 
(Chapter 5), there is still an explanation needed on how the VALIDATE approach 
can offer a legitimate, and reliable, way of answering policy relevant questions on 
health technology. This chapter attempts to provide such explanation based on the 
philosophical tradition of pragmatism. Specifically, it will offer a critical examination 
of the position of moral scepticism and suggest how it can be overcome by applying 
elements of pragmatism to the practice of health technology assessment (HTA). 
Especially the relation between pragmatism and the concept of interpretive frames 
(introduced in Chapter 3) will be clarified. 


Key message of this chapter: the philosophy of VALIDATE, partly based on pragma- 
tism, is that normative issues that arise in the context of HTA can be addressed in 
a rational way. 


After reading this chapter, you should be able to understand that the VALIDATE 


approach can best be conceived as a framework for evaluating health technology 
that is inspired by, but not synonymous to, pragmatist philosophy. 
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Introduction 


As already discussed in Chapter 1, a central tenet of the VALIDATE approach is 
that it regards HTA as a practice in which facts and values meet. There are two 
main reasons for this view: (i) a strict dichotomy between facts and values is unat- 
tainable and undesirable (Chapter 2); (ii) to answer policy relevant questions HTA 
should not only provide information on the plausibility of potential consequences 
of health technology, but also on the desirability of these consequences (Chapter 1). 
Answering questions on desirability requires being able to think about the impact of 
a health technology in light of a set of norms and values. Consequently, HTA needs 
to integrate empirical inquiry (collecting facts”) with normative analysis (‘values’). 
This enables HTA to explicitly take into account the needs of a population and deci- 


sion-makers, by exploring different conceptual schemes that stakeholders use 
in making sense of a concrete situation (Chapter 3 and Chapter 4). In addition, it 
increases awareness of the context-dependency of HTA (Chapter 6). 


To propose such a central role for normative analysis in HTA may invoke a critical 


response. A widely held view - also within the context of HTA - is that there is no 


way of rationally resolving normative questions. Therefore, trying to address these 


questions may be seen as a threat to the (scientific) legitimacy of HTA. This relates 
to a general discussion on the possibility of moral knowledge, which is also debated 


between ethicists (Chapter 5). Although there are methods for analysing and struc 
turing ethical arguments, there remains a worry about the status of the outcomes 


of such analysis and the experts involved in conducting them. These worries about 
the subjective nature of normative analysis are rooted in a general position of 
moral scepticism, which denies that there can be any form of moral knowledge or 
expertise. 


The challenge of ethical issues in HTA: moral scepticism 


Although the HTA community seems to be committed to addressing the wider 
ethical, legal and social implications (ELSI) associated with the use of health technol- 
ogies, alongside their safety, clinical effectiveness and cost-effectiveness, they are 
not always included in HTA reports that are drafted to inform healthcare author- 
ities. There are many reasons given for why ELSI issues are neglected or studied 


in isolation from other aspects of health technology, ranging from methodological 
to practical reasons (Bellemare et al, 2018). Although practical and methodological 
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obstacles are important to address, one fundamental challenge relates to how the 
HTA community pictures the natures of ethical issues. Ethical and societal concerns 
related to health technologies are often considered non-scientific, expressing 
controversial opinions that cannot be evaluated or supported with facts generated 
by scientific research. HTA agencies would then emphasize the importance of their 
reports being based on scientific facts, avoiding entering into possibly interminable 
‘oh, yes it is, oh, no it isn’t’ debates. This is a complex, but crucially important issue. 
For if the HTA community were to hold on to this distinction between (‘objective’) 
facts and (‘subjective’) opinions, it is not quite clear how it can take into account the 
wider ethical, legal and social issues associated with the use of health technologies 
(Legault et al, 2018). 


To make some headway in this complex issue, let's take a look at the debate on 
applied behavioural analysis (ABA; i.e., an intervention program, based on insights 
from behavioural science, aimed at enhancing, reducing and maintaining certain 
behaviours) for children with autism. The debate centres around the policy question 
whether ABA should be recommended (or used, offered, funded) as a treatment for 
children with autism. This question raises controversies for two reasons: (i) there is 
disagreement on the question whether ABA is an effective treatment in children with 
autism, and (ii) there is disagreement on the desirability of this treatment: what is it 
that we want to achieve by using ABA in children with autism? Should the aim be to 
help the child to recover from autism (assuming that such is possible), or should it 
focus on other aims (e.g., helping children with autism to cope with all sorts of social 


si 


tuations)? An example of the debate on its desirability can be seen in a statement 
made by a mother of two children with autism’: 


“Itis possible to change behaviour using ABA, there is no question of this - though 
not as effectively as some would have you believe. But this is beside the point. 


We ought to be considering whether we should change the behaviour - which is 
often harmless and often useful.” (Davison, 2018; emphasis added) 


The first part of this statement may be considered largely empirical: ‘It is possible to... 


The second part of the statement seems to be different, though, and more norma- 
tive in nature: ‘We ought to.... Here, the author challenges the view that changing the 
behaviour of children with autism is what we should be aiming at. The aim of the 


9 Davison, 2018, https://autisticuk.org/does-aba-harm-autistic people/. 
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treatment, normalization, is challenged. It is challenged, partly because it is consid- 
ered harmful. But also, because being autistic is considered as a condition that is 
not all wrong: 


“If | had the option of not being autistic, | would not take it - nor would | take 
it for my children. That does not mean life is easy for me, it means | like myself 
and my children the way we are.” (Davison, 2018) 


But if it is indeed a matter of disagreement on a normative issue, it is not immedi- 
ately obvious how this disagreement can be resolved. Indeed, it is even not obvious 
that it is settling what is called for, nor is it clear what we mean by it or how it should 
be achieved. A widely held view - also within the context of HTA - seems to be that 
there are no ways of rationally resolving such type of disagreements.'° This position 


is called moral scepticism, which refers to a diverse collection of views that deny or 
raise doubts about various roles of reason in morality (Sinnott-Armstrong, 2006)." 
This might explain why ethical concerns, such as those that have been expressed in 
relation to ABA for children with autism, fail to reach HTA reports that are meant to 
inform policy makers on the full spectrum of implications of such technologies. This 
could seriously hamper the consideration of ethical issues (or value issues more 
generally) in HTA. As a result, health technologies and the practices that are engen- 
dered by them seem to be completely separated from moral values. They occur, as 
it were, in completely separate, mutually inaccessible domains. This means that we 
need to redefine the mission of HTA (which invariably claims to include assessment 
of the wider ethical, social and legal implications associated with the use of health 


technologies), or we should reconsider our position towards the nature of ethical 
issues. 


10 This observation was made by David Banta and Bryan Luce as early as in their 1993 book, Health Care 
Technology and its Assessment (p. 152). 

11 Associated with this position of moral scepticism is the fear that an ethicist is imposing his (idiosyn- 
cratic, subjective, personal) values on others. It denies the possibility of moral expertise, to avoid this 
danger of value imposition. There is, of course, a strong relation between denying a role for reason in 
morality and the possibility of moral expertise. In Chapter 5 of this handbook we try to address this 
danger of value imposition by discussing methods for ethical analysis that can be used to evaluate 
moral arguments in a transparent way. For a discussion on the role of expertise in conducting ethical 
analysis in HTA, see also: Refolo P, Bond K, Bloemen B, Autti-Ramo |, Hofmann B, Mischke C, et al. Core 
competencies for ethics experts in health technology assessment. Int J Technol Assess Health Care. 
2020; 36(6): 534-9. 
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Rethinking the nature of ethical issues in HTA 


Different responses to moral scepticism 
If it turns out that there is normative disagreement on the desirability of a health 


technology, as in the case of ABA for children with autism, HTA agencies have several 


options: 


1. 


Adopting a strategy that can be referred to as the disclaimer strategy. 

This consists of proceeding with the HTA as originally planned but pointing out 
that the relevance of its outcomes is contingent on a particular interpretation 
ofthe policy problem, which does not seem to be shared by all stakeholders. 
The strategy might include the reporting of the scoping exercise (see Chapter 
4), pointing out in what respect the interpretations differ, and how that might 
affect the choices (and outcomes) of the HTA. Although adopting such a 


strategy would not result in a different HTA, it at least testifies of an awareness 
of the contingency of its outcomes, and commissioning organizations that need 


to make decisions on the basis of the HTA can take this into account. 


. Asecond strategy might be called the justificatory strategy. It differs from the 


disclaimer strategy in that it involves a critical appraisal of the results of the 
scoping exercise by the HTA team. Critical appraisal, here, would not so much 
refer to a critical appraisal of the evidence of safety and clinical and cost-effec 
tiveness, but to a critical appraisal of background theories and normative pref- 
erences (see Chapter 3), 


> 


equiring different methods of appraisal. On the basis 
of the results of such an analysis, an HTA agency, preferably in conjunction with 
the commissioning organization, could decide which interpretations are consid- 


ered sufficiently plausible and reasonable to inform the choices regarding the 
HTA. This would help to make those choices explicit and justify them towards a 
wider audience. 


. The third and last option is the strategy of HTA as learning (Grin et al, 1996), 


Here, the agency will conduct a critical analysis of the interpretive frame of 
stakeholders (similar to the second strategy). In case of multiple, mutually 
incompatible frames, it would feed back the results of the analysis to the 
various stakeholders in an attempt to achieve a certain degree of learning 
among them. The learning would not necessarily be directed to find out who 
(or what) is right and who (or what) is wrong. Rather, participants may come to 
realize the complex nature of the problem, and how factual issues and value 
issues are intertwined and jointly appear to define what is considered relevant 


to the assessment. Also, they may come to accept the indeterminacy of certain 
issues, and be willing to suspend, at least provisionally, their judgement and 
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accept the relevance of the issues to be addressed in the HTA (its terms of 
reference). This approach has been described by Guba and Lincoln (Guba et al, 
1989) and applied in the context of HTA to the use of cochlear implants in deaf 
children (Reuzel, 2002). Clearly, because of the involvement of stakeholders, 
this strategy may be more effective in terms of resolving long-standing contro- 
versies, but is likely to be more time-consuming too. lt stands to reason that 


HTA agencies and commissioning organizations jointly decide which ofthese 
approaches seems most appropriate and feasible in concrete cases, given limi- 


tations in time, capacity, and resources. 


The VALIDATE approach adopts the latter option (HTA as learning”), which also 
challenges the position of moral scepticism. Our main concern is whether we can 
somehow re-conceptualize the relationship between facts, values and technologies, 
in such a way that the sorts of concerns that have been expressed regarding the use 
of ABA in children with autism can be incorporated in HTA in a more productive and 


satisfying way. 


The status and meaning of ‘ethical issues’: the case of Extra- 
Corporeal Membrane Oxygenation (ECMO) 

The VALIDATE approach differs from mainstream HTA in its conceptualization of 
‘ethical issues’. In current HTA, ethical issues are considered alongside issues such 
as Safety, clinical effectiveness, and cost-effectiveness. Such classification runs the 
risk of obscuring both the moral significance of the safety, clinical and cost-effective- 
ness claims and the empirical content of ethical issues associated with the use of 
health technology. To explain this further, consider the example of Extra-Corporeal 
Membrane Oxygenation (ECMO, or heart-lung machine, see Figure 1) in new born 
children with conditions such as meconium aspiration or diaphragmatic hernia (Crow 


et al, 2009). Outcomes that have been reported in the literature include complica- 
tions that result from intracranial haemorrhage, improved survival, and cost-effec 
tiveness of ECMO when compared to optimal conventional support. Ethical issues 
that have been reported to be associated with the use of ECMO in new born chil- 
dren include the compromise of parental autonomy and the dilemmas associated 
with the discontinuation of ECMO support in case the new born child fails to thrive 
(Kirsch et al, 2018, Williams et al, 2016). This distinction seems to suggest that there 


are, on the one hand, outcomes that can be objectively established (safety and 
clinical and cost-effectiveness), and, on the other hand, outcomes for which this 
does not hold (ethical issues). 
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Figure 1 Schematic illustration of Extra-Corporeal Membrane Oxygenation 


(ECMO) — Source: Júrgen 


Schaub. de:User:Mr.Flintstone [CC BY-SA 2.0 DE. Available 


from: https://creativecommons.org/licenses/by-sa/2.0/de/deed.en] 


The current practice in 


HTA of distinguishing ethical issues from other issues in 


this way is, however, dubious. For it tends to obscure the fact that both categories 


of outcomes are morall 


y relevant and rooted in an empirical reality of a specific 


practice (Hofmann et al, 2018). For instance, the issue of complications derives its 


relevance from a comm 


itment to avoid inflicting harm. Likewise, improved survival 


derives from a commitment to doing good, and cost-effectiveness from a commit- 


ment to distributive justice. At the same time, the issues of parental autonomy and 
discontinuing ECMO support are based on a particular, real-life practice of care of 


newborn children with li 


fe-threatening conditions. 


Box 1 The status and meaning of “ethical issues': the case of Non- 
Invasive Prenatal Testing (NIPT) 


This mainstream, and problematic, way of conceptualizing ethical issues 
in HTA can also be seen in assessments of non-invasive prenatal testing 
(NIPT). Ethical issues concerning NIPT (e.g., which conditions should be 
tested? When should it be offered? Is it a medical procedure?) are often 
addressed in separate analyses from studies that assess the safety and 
validity of NIPT (Kibel et al, 2017). This ignores that an answer to the cen- 
tral normative question related to NIPT, what is its desirable purpose, is 
already implied by any analysis of its validity and cost-effectiveness. To be 
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able to assess the reliability of NIPT you need to decide on which genetic 
conditions it should offer information. And any cost-effectiveness analysis 
of NIPT needs to define its potential benefits and costs, which raises nor- 
mative questions concerning whose benefits should be taken into account 
(e.g., the QALYs of the unborn child or the parents) and which time hori- 
zon should be considered (e.g., should only impacts up to the detection 
of affected pregnancies be taken into account, or also long-term impli- 
cations such as reduction in the use of healthcare resources by children 
born with trisomy 21). This shows that both ethical and clinical analyses of 
NIPT are rooted in normative commitments. In addition, assessing claims 
on potential consequences of NIPT, like that it enhances reproductive 
autonomy, requires both value-laden decisions on how to define certain 
outcomes and empirical inquiry to understand their particular meaning 
and realization in the context of NIPT (Bloemen et al, 2021). 


The status and meaning of ‘ethical issues’: the VALIDATE approach 
By introducing the interpretive frame as a central concept in evaluation 
(Chapter 3), the VALIDATE approach abandons the current distinction 
between objective (factual) and subjective (normative) issues. Here, rele- 
vant aspects of a particular practice (for instance, the care of newborn 
children with life-threatening conditions) emerge from a commitment to 
and working knowledge of certain moral principles (e.g., of not causing 
harm, of respecting autonomy, of observing justice), in conjunction with 
a certain knowledge about this practice and its outcomes. Hence, we do 
not wish to suggest that facts and values cannot be distinguished from 
each other. We can make a distinction. But the point is that in HTA, facts 
and values act in conjunction, enabling us to acknowledge which aspects 
of a situation, event or action seem to be relevant and need to be taken 
into account when we wish to assess how and why they are meaningful to 
us. In the following section of this chapter we will elaborate this further 
by exploring the relation between the VALIDATE approach and the philo- 
sophical tradition of pragmatism. 
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Pragmatism and the VALIDATE approach 


Interpretive frames 

As stated above, our main concern is how we can re-conceptualize the relationship 
between facts and values, in such a what that the sorts of concerns that heave been 
expressed regarding the use of ABA in children with autism can be incorporated in 
HTA in a more productive and satisfying way. For this, we turn to the work by Frank 
Fischer. Fischer developed a model where judgments of specific solutions (e.g., is 
ABA an appropriate sort of treatment for children with autism?) are closely linked to 
how a problem is defined (e.g., what ae the problems that are caused by autism?), and 
where both, judgments of solutions and problem definitions, are informed by back- 
ground theories (e.g., what sort of long term impact can be expected from teaching 
people to behave differently form how they are?) and normative preferences (e.g., 
is it generally acceptable to try to normalize people?). As stated before (Chapter 3), 
this set of judgment of solution, problem definition, background theory and norma- 
tive preferences is referred to as an interpretive frame. This can be considered a type 


of tacit knowledge: it remains mostly implicit, but it can be made explicit. The method 


for this is called reconstructing interpretive frames (Chapter 3). A key feature of this 
method is that it does recognize that facts and values can be analytically distinguished 
(e.g., a distinction is made between background theory and normative preferences), 
but at the same time it emphasizes that they act in conjunction, determining what is 
being perceived as problematic (problem definition) and what is considered both 
a feasible and appropriate way of resolving the problem (judgment of solution). By 


reconstructing the interpretive frames of multiple stakeholders, differences in judg- 
ments of specific solutions (such as ABA in the treatment of autism) can be related to 
differences in problem definition, background theories and normative preferences. 
The purpose of making the factual and normative assumptions underlying judg- 


ments on technologies explicit is to enable a critical, constructive scrutiny of those 


assumptions. HTA experts can play a novel and significant role here. Its major aim is 
to facilitate learning among the various stakeholders, generating new conceptualiza- 
tions, perspectives, and approaches to resolve problems. 


Interpretive frames and pragmatism 

So, the interpretive frame is one of the key elements of the VALIDATE approach. 
In Figure 2, the four elements of the interpretive frame are presented once more. 
Problem definitions are in the top left quadrant, judgments of solutions are in the 
top right quadrant, and background theories and normative preferences are in the 
lower left and right quadrant, respectively. 
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PROBLEM DEFINITION JUDGMENT OF SOLUTIONS 
What are the key problems of Is this technology likely to be effective, 
the current situation? appropriate, feasible in th 


is context? 


BACKGROUND THEORY NES NORMATIVE PREFERENCES 
What mechanisms are responsible for What is it that we would like to achieve 


the current burden of disease? and what is it that we would 


Figure2 Interpretive frames 


like to avoid? 


ons wali lulels| 


This arrangement enables us to think of the interpretive frame as being constituted 


of two axes, and we can ask: what do these two axes represent? What do problem 


definition and background theory have in common that distinguishes them from 


judgments of solutions and normative preferences (the horizontal a 
we can ask what problem definition and judgment of solutions have in 


xis)? Similarly, 
common that 


distinguishes them from background theory and normative preferences (vertical 


axis)? To answer these questions, we would suggest the following characterization 


of the two axes: the horizontal axis represents an axis which runs 
descriptive (to the left) to primarily prescriptive (or normative) ( 


acting, while the lower halfis primarily related to reflection, to thinking 


Apart from the question how the two axes might best be characterized, with thi 
arrangement of the elements of interpretive frames we can also ask: how do th 


two poles on each axis relate to each other? Our answer to that ques 


that they mutually constitute one another. In other words: they need, determine, an 


presuppose one another. For the vertical axis this means that we wou 
to think in the way we do, if we were not able to act in the way we do, a 
The nature of our actions would be quite different if we were not able 
way we can; similarly, the nature of our thinking would be different i 


rom primarily 
to the right). 


The vertical axis represents an axis which runs from more general (or theoretical, or 
abstract) (the lower half) to more specific (or practical, or concrete) (the upper half). 


This can also be held to mean that the upper half is primarily related to practice, to 


(see Figure 3). 


Ss 
e 
tion would be 
d 
e 


d not be ab 
nd vice versa. 
to think in the 
f we were not 


able to act in the way we can. This idea of the close entanglement of thinking and doing 


is inspired by pragmatism. 
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Figure 3 Interpretive frames with proposed axes 


Pragmatism is a tradition in philosophy which originates from the work of a number 
of American philosophers in the second half of the nineteenth and first half of the 
twentieth century.'? One of the common themes in their thinking was the idea that 
one of the main ways (if not the way) in which we acquire knowledge is through 
acting. The theories and concepts that we have are tools that enable us to act in 
a certain way. For example, that “bear” refers to a furry creature with teeth is not 
an inherent or essential property of this term or the description of this particular 
creature, but people found this syllable useful for pointing out dangerous crea- 
tures and helping us to survive. So, if we wish to know whether some statement 
is true, we need to imagine what course of action would be most reasonable if we 
assume that the statement is, in fact, true. Also, we would need to establish what 
would follow, or what we would expect to happen, when we were to act along those 
lines. If our expectation is, in fact, borne out, we may be relatively confident that the 
statement is true.’ Acting, then, makes knowledge possible. However, pragmatists 


12 For more information on pragmatism, see also: Catherine Legg and Christopher Hookway 2019, 
https://plato.stanford.edu/archives/spr2019/entries/pragmatism. 

13 Although we do consider this focus on the close entanglement between knowledge and practice a 
strength of pragmatism, we should also be attentive of its limitations. Firstly, it means that prag- 
matists are unlikely to be particularly interested in knowledge that does not seem to have obvious 
practical implications, After all, in the pragmatist view, there is no way of ascertaining whether such 
propositions are likely to be true (or false). This might impose an undue restriction on the sorts of 
knowledge that seem to be worthwhile to pursue. Also, we have to bear in mind that the issues that 
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would be quick to point out that complete certainty is unavailable; our knowledge is 
always incomplete, provisional, and subject to refutation (a position that is usually 


ot 


referred to as fallibilism). However, this does not relieve us from the obligation to 
always be truth-directed, or take an interest in seeking the truth, even though we 
know that it is an unattainable goal.'* At the same time, a commitment to fallibilism 
should make us receptive for alternative interpretations of particular situations or 
events, provided that there is a shared commitment to think through and test the 
practical consequences of competing interpretations. This, in fact, holds for both 
domains, the descriptive and the normative domain: acting under the direction of 


background theories and normative preferences should reveal whether our general 


notions are likely to be correct. While, in the view expressed here, action makes 
knowledge possible, the reverse also holds: a particular way of acting (rationally) is 
enabled by us having particular knowledge. This is a distinctive feature of pragma- 
tism: is describes an iterative process between thinking and doing. 


A similar relation holds between the two poles on the horizontal axis, which runs 
from primarily descriptive to primarily prescriptive. Our knowledge and experience 
of how the world is (description) and our knowledge and experience of how the 
world might be and might be a better place (prescription), constitute one another. 
Our capacity of being aware of the world existing in a specific way is dependent 
on our capacity of being aware of the fact that the world could, in some ways, also 
be otherwise. Moreover, we have the capacity for not being neutral, or indifferent, 
toward those different possible worlds.'? In other words: we would experience our 
life world in a completely different way if we were not, at the same time, able to 
imagine that our life world could, at least in some respects, also be different, and 
that humans can, at least to some extent, affect their life world (that is: it is not, or 


the founders of pragmatism had in mind when reflecting on the nature of knowledge and its associ- 
ation with practice, are rather different from the issues that are nowadays on the agenda. A case in 
point would be the current debate on global climate change. The complexity of many current issues 
may defy the thinking through of what sort of actions would be most reasonable, if we assume that 
our account of them is correct; similarly, complexity may defy a straightforward interpretation of what 
is being observed in such cases. Finally, some issues may require a long time of follow up in order to 
discover the full range of implications. By pointing to these limitations, we do not wish to suggest that 
the pragmatist account of knowledge is no longer of use to the present context; rather, these issues 
need to be addressed in order to see whether and how the classical pragmatist account is still relevant 
for the complex issues that we are typically facing in the 21st century (see, for instance, Sanderson 
|. Intelligent policy making for a complex world: pragmatism, evidence and learning. Political studies. 
2009; 57(4): 699-719 

14 Pragmatism has been characterized as accepting fallibilism, without embracing skepticism; Bacon M. 
Pragmatism: an introduction. Cambridge: Polity Press; 2012. 

15 See for instance Sayer A. Why things matter to people: Social science, values and ethical life. New York: 
Cambridge University Press; 2011. 
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not entirely, necessarily the way it is). What would drive humans to explore their life 
world in order to find out how things work, if there is not, at the same time an idea 
(even emergent) that such knowledge might be used in order to change this life 
world (for the better)? And, vice versa, how can notions of how the life world may 
be changed develop in the absence of knowledge on how the life world is, how it 
works? Thus, facts (the descriptive) and values (the prescriptive) can be meaningfully 
distinguished; at the same time, however, these two realms mutually constitute one 


another. Acknowledging this close entanglement of facts and values is also a distinc 
tive feature of pragmatism. 


Conclusion 


Having read this chapter, you should be able to understand that the VALIDATE 
approach can best be conceived as a framework for evaluating health technology 
that is inspired by, but not synonymous to, pragmatist philosophy. It acknowledges 
the importance of empirical analysis in this context but denies that HTA is merely a 
matter of collecting the facts about the consequences of health technology. Rather, 


it holds that HTA is a matter of collecting facts that are considered relevant, plau- 
sible, and amenable to generally accepted methods of inquiry (Grin, 2000). This refor- 
mulation reveals the normative nature of HTA. For relevance is determined by our 
commitment to, and interpretation of, values. Plausibility is determined by our 
understanding of reality (or ontological commitments). And amenability to methods 
of inquiry is determined by our methodological commitments: what types of inquiry 


(i.e. research, analysis) do we consider to likely yield valid and reliable knowledge 
of the world? Reconstructing these frames provides a way of putting HTA and its 
results in a wider framework and a means for better understanding and construc 
tively exploring the interdependence between these different types of normativity 
and empirical inquiry. lt recommends exploring the contents of interpretive frames 
as an integral part of HTA. To be sure, this need not always reveal deeply divergent 
views on the nature of the problem, on the type of solutions that are considered 
appropriate and the sort of standards by which these should be judged. But it can, 
in all cases, provide a more satisfying way of justifying the choices that are, inevi- 
tably, made in HTA. As such, it can help to bolster the accountability of the processes 
used by governments to decide technology may (not) be used (e.g., decisions on the 
introduction, use, reimbursement or disinvestment) to improve the living conditions 
of its citizens. 
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Summing up 
VALIDATE 


Abstract 


Inthe preceding chapters ofthe handbook, key elements ofthe VALIDATE approach 
to health technology assessment (HTA) have been elaborated. In this concluding 
chapter, we will bring these elements together in an overarching framework. In the 
VALIDATE approach HTA is proposed as it was originally intended: as a type of policy 
analysis, wherein the study of safety, clinical and cost-effectiveness of (new) health 
technologies and their wider ethical, legal, and social implications are closely inte- 
grated, and stakeholders are involved in a more productive way throughout the 
process of HTA. In addition, we will highlight what skills and knowledge are needed 
to use the VALIDATE approach in practice. 


Key message of the chapter is that by subsuming HTA in a wider process of prac- 
tical reasoning about the proper use of health technologies, the VALIDATE approach 
holds the promise of further enhancing the policy relevance of HTA. 


After reading this chapter, you should be able to explain what the VALIDATE approach 
to HTA is, and what skills and knowledge HTA-doers need for conducting HTA in this 


vein. 


Stakeholder views, empirical research, and normative 
inquiry 


In this handbook we present and discuss the VALIDATE approach (see Figure 1) 
that is focused on how to integrate empirical research (facts) and normative inquiry 
(values) when assessing health technologies (HTA). To exemplify this, we used non-in- 
vasive prenatal testing (NIPT) as a case study. NIPT is a prenatal screening procedure 
that analyses cell-free foetal DNA, which circulates in the mother’s blood during 
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pregnancy, in order to obtain information about the foetal genotype (see Chapter 1 
for an introduction of the case study). 


Stakeholder involvement 
(e.g., patients, hospitals, payers, clinicians) 


exchange 


Values (e.g., prefer- 
ences, equity, fairness) 


exchange 


Data (Research) 


VALIDATE: integrative HTA process 


Analysis Recommendation 
(Assessment) (Appraisal) 


Figure 1 The integrative approach of VALIDATE 


In this chapter, we summarize the key elements of the VALIDATE approach to HTA that 
have been elaborated in the preceding chapters. We specifically ask what generic 
lessons may be learned from NIPT as a particular case for HTA: Should the sort of 
views, held by or expressed on behalf of some of the relevant stakeholders involved 
(e.g., parents, unborn child), be taken seriously? Why, or why not? (or perhaps rather: 
when, and when not?) What does it mean, exactly, to take stakeholders’ views seri- 
ously in this context (of introducing/using NIPT)? And, finally, what is it, exactly, that 
we hope to learn from stakeholders, and how should we conduct and organize an 
HTA in order to make this happen? 


To address these questions, we propose that a key objective of HTA, consistent with 
its recent definition (O’Rourke et al, 2020), is to help communities (local, regional, 
national or supra-national) to develop a shared view of how specific health problems 
can best be resolved in their specific context, and what role health technologies can 
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fulfil in such endeavour. An inquiry to this end will, of course, include considerations 
of benefits and (opportunity) costs that are associated with the use of health tech- 
nologies, but may extend to other considerations as well. 


Specifically, ascertaining a proper role for health technologies in resolving particular 
health problems can substantially benefit from an understanding of the major 
contributory causes of the health problem, of the physical and cultural context, of 


potential options for resolution, and ofthe experiences that have been obtained with 
these options. Since HTA is an evaluative exercise, the inquiry is guided by, on the 
one hand, considerations of what is possible, and, onthe other hand, considerations 


of what is desirable and acceptable. As such, the inquiry can be characterized as an 
instance of practical reasoning: figuring out what needs to be done in a concrete 
situation, given a commitment to a range of general, normative (e.g., ethical) princi- 


ples (Fischer, 2007). Such an inquiry is, we would suggest, inherently ambiguous. By 
this, we mean to say that there is no procedure that, when followed correctly, can 
unambiguously reveal the right way to act or proceed. This ambiguity results from 
at least two different sources. Firstly, it results from the necessarily provisional and 
incomplete nature of the knowledge that is needed in order to decide how to act.A 
second source of ambiguity results from properties of general ethical principles (see 


Chapter 5). Following Richardson (1990; 1997), we hold that such principles can best 


be considered as incommensurable. It means that there is not a single metric (such 
as utility) to which the various ethical principles can be reduced (see also Chapter 5). 


Taking this more integrative perspective on HTA, what we would hope to hear from 
stakeholders, then, are particularly their views of how specific health problems may 
best be resolved, and what role they see for health technologies. Not, to be sure, 
as some form of sacred truth. But primarily as a specific perspective, a perspective 
that should make us think, or rather: re-think, and urge us to explore the reasons 
underlying those views, as well as the reasons underlying our own views. Such a 
perspective is also likely to point to essential aspects of the context (social and phys- 
ical) that need to be taken into account (see Chapter 6). 


As explained in Chapter 3, the method of reconstructing interpretive frames can 
be usefully applied to reveal how such judgments are related to assumptions about 
the nature of the problem (background theory) and about the sort of things that 
ought to be pursued or avoided (ethical commitments). As further explained in 
that chapter, making such connections explicit renders them amenable to critical 
and constructive scrutiny. The inquiry should, then, help those who are involved 
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to identify and acknowledge ambiguities, uncertainties and apparent conflicts, and 
that may occasion them to re-think their views and suspend their judgement. In 
this sense, the VALIDATE approach may be considered as an approach to HTA that 
urges the various stakeholders to avail themselves of the opportunities for mutual 
learning, offered by the inquiry (Grin et al, 1996; Grin et al, 2007; Richardson, 2016). 


Making it work in practice 


In addition to the knowledge and skills (procedural ad attitudinal) that are currently 
taught in HTA curricula across the globe (Mueller et al, 2020), the HTA-doer would 
be well advised to develop the skill of identifying the relations between health 
technology and its associated primary studies on the one hand, and the under- 
lying assumptions on the other hand. This means that the task of the HTA doer is 


not only to identify those who may be considered stakeholders and solicit their 
claims and concerns (Guba et al, 1989) regarding the health technology, but also 
to assess whether these claims and concerns can serve as a basis for public deci- 
sion-making and action. It is for this reason that the method of reconstructing inter- 
pretive frames plays such a key role in the VALIDATE approach (see Chapter 3). It 
can help to elicit judgments from stakeholders regarding various approaches to 
resolve a specific health problem, and to relate those judgments to problem defini- 


tions, background theory and ethical commitments. By conducting a critical exam 
ination of their content (e.g., evidential support, coherence with knowledge and 
ethical commitments for which broad support in the relevant community may be 
presumed), HTA doers can indicate to decision and policy makers whether and why 
(i.e., on what grounds) they can serve as a legitimate basis for public policy making 
(the flip side being that it might be hard to justify not to take them into account in 
public decision-making). 


A sequence of activities for HTA doers that can be derived from this, is, then: 

= Identify stakeholders: given the nature of the health technology and the nature 
of the health problem that are the focus of the HTA, who can be designated 
as stakeholders (i.e., those affected by a decision and who are able to bring a 
specific perspective on the issue to the table)? 

=" Elicit views (judgments) from stakeholders regarding the subject of the HTA. 


= Identify underlying assumptions: what assumptions appear to underlie those 
views? Ensure to address assumptions regarding what is considered to be the 
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case, that is, how things are, work, and are related to each other, as well as 
assumptions regarding what is considered desirable and what is best avoided. 

= Conduct a critical analysis: can the views that were elicited and their underlying 
assumptions jointly (that is, as a whole) stand up to scrutiny? 


Provisionally define and negotiate the terms of reference of the HTA: given the 
outcome of the analysis so far, what would be the key questions to be addressed 
in the HTA, and how might they best be explored? To ensure a sense of owner- 
ship and shared responsibility, this step should preferably be conducted in close 
collaboration with stakeholders and with the HTA commissioning organization. 
The latter will usually set constraints on the process on the basis of available 
time and resources and on the basis of its (sometimes self-professed) remit. 
Occasionally, defining the terms of reference of an HTA may require a delicate 


trade-off between feasibility and policy relevance. 


Jointly, we would consider these steps the core of an HTA scoping exercise (see 
Chapter 4). Clearly, stakeholders have a serious role to play in the process, and the 
n 


method of reconstructing interpretive frames broadly defines the nature of the 


input that is sought from them. When, how, and how extensively stakeholders are 
involved in the HTA process may vary from case to case. Saturation can serve as a 


useful criterion to assess comprehensiveness of the effort (see, for instance, Saun- 
ders et al, 2018). 


When adopting the VALIDATE approach to HTA, it stands to reason that HTA doers 
declare, as part of their reporting, who were designated and actually involved as 
stakeholders, in what way they were involved, and what claims and concerns were 
solicited in the process. They would also clarify what assumptions they identified 
as underlying those claims and concerns, how they assessed the validity of those 
assumptions, and the key findings of their assessment. In other words, they report 


the outcomes of the scoping exercise, and indicate what selections were made of 
topics that were included in the HTA. 


Broadening the options for decision-makers 


If specific ways of using a health technology were found to be based on assump- 
tions that are sufficiently supported by evidence, coherent with current knowledge 
and consistent with general ethical principles that may be presumed to obtain, this 
would enable decision-makers to decide whether they are actually willing to act 
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upon those findings and assumptions. They can indicate how the health technology 
is to be used in their jurisdiction, indicate which general ethical principles are thus 
expressed, and how potential conflicts between general principles were dealt with. 
Provided that the process of stakeholder involvement and subsequent analysis has 
worked well, the resulting decision may also be expected to be sufficiently reflective 
of viewpoints that are held in the relevant community. On the basis of the results 
of the HTA, decision-makers could also decide to identify any gaps, uncertainties, 
inconsistencies or conflicts in currently available evidence or interpretations thereof 
and suggest ways of addressing those in future studies. In a similar vein, they can 


indicate if any measures will be taken to monitor whether the proposed use of the 


health technology works out the way it is expected to do. 


Broadening the outcome of HTA in this way would, in our view, further enhance the 
relevance of HTA to policy- and decision-making. Clearly, however, HTA doers need 
to discuss with commissioning organizations whether they would endorse adoption 
of a VALIDATE approach to HTA. If anything, it will require a more iterative process, 


discussing at various stages findings and ways to proceed, given available time and 
resources. It stands to reason that the HTA team and commissioning organization 
jointly decide on which of these issues are, in fact, included in the HTA. 


A comparison with current practices of HTA 


So how does the VALIDATE approach differ from currently prevailing practices of 
HTA? Current approaches run the risk of being of limited use in squarely facing 
ethical dilemmas that are posed by technological developments and attaching 
practical consequences to this. By collaborating with stakeholders in developing a 
range of policy options and critically analysing those options for empirical support 
and congruence with general ethical principles, the VALIDATE approach holds the 
promise of offering a wider range of options for decision-makers. Here, ethical 
issues, as assumptions regarding what is desirable, are on a par with assumptions 


regarding what is the case, jointly and in mutual interaction providing guidance to 
the inquiry (see Chapter 2; Fischer 2007; Lindauer 2020; Van der Wilt, 2005). 


One should note that there may be conflicts between the perspectives of different 
stakeholders, generating controversies with respect to goals, outcomes, and 
evidence used in HTA. While the VALIDATE approach may help (re)solving or recon- 
ciliating these, there may be basic conflicts that cannot be addressed. However, 
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the VALIDATE approach provides an open, transparent, and accountable approach 
that specifies this. It makes the value issues and judgments as well as stakeholders’ 
perspectives more explicit than contemporary HTA. 
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Glossary 


Axiology 


Background 


theory 


Coherentism 


Congruency 


Context 


iterature referred to 


The definitions provided here are mainly derived from the international HTA glos- 
sary, official collaboration between International Network of Agencies for Health 
Technology Assessment (INAHTA), Health Technology Assessment international 
(HTAi) and other partner organizations: http://htaglossary.net/HomePage; and key 


in this handbook. 


The philosophy or theory of values. A branch of ethics 


The set of (interrelated) normative and empirical propositions that 
shape observations and interpretations of an actor; constitutes one 


layer in an actor's interpretive frames 


The belief that an ethical proposition or belief is true and justified 
if it is part of a network or set of ethical beliefs people hold that are 


jointly coherent 


A pragmatic, action-oriented and contextual form of agreement 
about an issue. We speak of congruency if a problem definition and 
matching solution have been found that have a sensible meaning 
each of the actors involved in the issue, to such an action that 

joint action is conceivable. As different actors may have different 
interpretive frames these meanings may differ; quintessential is that 


they yield action that contribute the joint effort 


The conditions and circumstances in which an intervention is 
applied. For example, the setting (in hospital, at home, in an air 
ambulance), the time (working day, holiday, night-time), the type 

of practice, (primary, secondary or tertiary care, private practice, 
insurance practice, charity). However, may also include geographical 
characteristics, epidemiological characteristics, socioeconomic, 


sociocultural, political, legal and ethical considerations 
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Coproduction 


Epistemology 


Euthanasia 


Evidence- 
informed 
deliberative 


processes (EDPs) 


Fact 


Foundationalism 


Health 


determinants 


Health 
technology 


Health 
technology 
assessment 
(HTA) 


Process in which policy objectives are realized between policy 
makers and actors involved in policy implementation (a policy's 
target group). It is usually enabled and promoted by policy 
instruments that help and incentivize the latter actors to act in line 


with policy objectives 


The philosophy of knowledge and how we gain knowledge 


Actively to inject for example potassium chloride in a patient with the 


intention that the person should die 


A practical, stepwise approach for HTA bodies to enhance legitimate 
benefit package design, based on deliberation between stakeholders 
to identify, reflect and learn about the meaning and importance of 


values, informed by evidence on these values 


Something that is true, something that can be verified according to 


an established standard of evaluation 


The belief that there are some values and norms that are self- 
evident, which we cannot doubt as true and justified, and which we 


can build upon 


Combination of factors that affect the health of individuals and 
communities. These factors include the social and economic 
environment, the physical environment, and the person's individual 


characteristics and behaviours 


An intervention developed to prevent, diagnose, or treat medical 
conditions; promote health; provide rehabilitation; or organise 
healthcare delivery. The intervention can be a test, device, medicine, 


vaccine, procedure, program, or system 


A multidisciplinary process that uses explicit methods to determine 
the value of a health technology at different points in its lifecycle. 
The purpose is to inform decision-making in order to promote an 


equitable, efficient and high-quality health system 
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Hospital-based 
decisions 


Interpretive 


frame 


Judgement 


Macro-level 


decisions 


Meso-level 


decisions 


Meta-level 


decisions 


A specificity of meso-level decisions. This refers to decisions at 
the managerial level within a specific hospital. The information 
and facts collected for those decisions are “in” and “for” hospitals. 
Hospital based decisions consider the level of complexity of the 
hospital, the structure, the financial capacity (budget), the skills of 
the professionals, the characteristics ofthe patients that could be 


admitted and the society within the hospital is embedded 


Perspective on a particular issue in a particular context) comprising 


two layers that are specific to that issue (the problem definition and 


solution assessment; and two layers with the more generic beliefs 
that form the lens through which the issue-in-context is perceived: 


background theories and normative preferences 


The action-oriented, contextual process of finding for a particular 
issue a problem definition and solutions that match it. This process 
essentially iterates between considering the problem and defining 


associate solutions which the actor sees as feasible and acceptable 


It concerns decisions on allocation and utilization of resources in a 
region, organization, hospital, etc. This level uses instruments that 
allow policymakers to establish priorities when allocating funds for 


different healthcare programs or single interventions 


It pertains to the clinical decisions that can be used for the treatment 
of specific conditions or for groups of patients with similar diseases. 
Optimal treatment policy for groups of patients with similar clinical 
characteristics. This level uses instruments that allow clinicians to 
select treatments that have been shown to provide the best overall 


outcome for groups of patients affected by the same disease 


At this level, politicians and managers make decisions regarding 
the healthcare of large populations balancing the financial and 
humanitarian aspects of service delivery. For example, in countries 
where healthcare is public, politicians need to decide what 
percentage of the total revenue will be delivered for health-care 
among all the other competing needs, e.g., education, defence, 


infrastructures, research etc. 
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Micro-level 


decisions 


Multi-criteria 
decision analysis 
(MCDA) 


Naturalistic 


fallacy 

Nihilism 
Non-Invasive 
Prenatal Testing 


(NIPT) 


Normative 


preferences 


Ontology 


Phronesis 


Principlism 


Problem 


definition 


It concerns decisions on allocation and utilization of resources in a 
region, organization, hospital, etc. This level uses instruments that 
allow policymakers to establish priorities when allocating funds for 


different health-care programs or single interventions 


An umbrella term to describe a collection of formal approaches 
which seek to take explicit account of multiple criteria in helping 


individuals or groups exploring decisions that matter 


The supposed fallacy of inferring evaluative conclusions from purely 


factual premises 
The belief that all values are baseless and that nothing can be known 
A procedure for gaining information about a foetal genotype by 


examining samples of foetal DNA obtained from the mother's blood 


Fundamental ontological views, forming one layer in an actor's 
interpretive frames; for a medical professional these may include 
the determinants of health and disease seen as central, and the own 


identity viz-a-viz patients, policy makers and others 


The philosophy of existence, i.e., of being (of things) 


Type of wisdom or intelligence relevant to practical action, implying 


both good judgement and excellence of character and habits 


A mid-level theory in normative ethics that believes that we share 
four basic ethical principles that can be applied to solve moral 
problems: respect for autonomy, non-maleficence, beneficence, and 


justice 


The way in which a specific actor gives meaning to the situation / 
make sense of an issue as a discrepancy between the state the actor 
observes or anticipate, and what the actor considers desired (future) 


state 
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Problematic 


conditions 


Quality-adjusted 


life year (QALY) 


Realism 


Scoping 


Solution 


assessment 


Stakeholder 


Stakeholder 


involvement 


Value 


Observable features of the situation that co-constitute an issue) 


A unit of outcome of an intervention where changes to years of life 
subsequent to this intervention are adjusted according to the quality 


of life during those years 


The belief that phenomena (such as things) in the world exist 


independent of human beings experiencing them 


Defining the objective and research questions of an HTA by 
a systematic exploration of relevant aspects from multiple 
perspectives (e.g., patients, informal carers, health professionals, 


decision-makers) 


How an actor sees and weighs (side-) effects and costs of specific 


solutions; constitutes one layer in an actor's interpretive frames 


An individual with an interest in the outcome of the HTA process final 


decision 


An iterative process of actively soliciting the knowledge, experience, 


judgment and values of individuals selected to represent a broad 


range of direct interest in a particular issue, for the dual purposes of: 
creating a shared understanding, making relevant, transparent, and 


effective decisions 


The regard, merit, importance or worth given to something. It is the 


basis for showing a preference i.e., making a choice 
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Health technology assessment (HTA) is defined as a multi- 
disciplinary process that uses explicit methods to determine 
the value of a health technology at different points in its 
lifecycle. The purpose is to inform decision-making in order 
to promote an equitable, efficient, and high-quality health 
system. The definition reflects that facts and values are 
intertwined in HTA. 

This means that HTA should be considered as atype of policy 
analysis, wherein the assessment of safety, clinical and cost 
implications of health technologies, as well as their wider 
ethical, legal, social, organizational, environmental and other 
implications is conducted from the view that these aspects 
are closely interrelated, and wherein stakeholders are in- 
volved in a more productive way throughout the process of 
HTA. Acknowledging this holds the potential of conducting 
assessments of health technologies in a way that supports 
deliberative democratic decision making. 

In the 2018-2021 EU Erasmus+ strategic partnerships project 
“VALues In Doing Assessments of healthcare TEchnologies” 
(VALIDATE), a consortium of seven academic and HTA organi- 
zations have developed an approach to HTA that allows for 
the integration of empirical analysis and normative inquiry. 
The VALIDATE handbook: an approach on the integration of 
values in doing assessments of health technologies offers the 
reader an opportunity to get acquainted with the theoretical 
considerations and apprehend the associated practical 
and organizational implications of this approach. It offers 
those interested in HTA to integrate empirical analysis and 
normative inquiry in a transparent way. 
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